[Bug 2108994] Re: apscheduler retrieving decimal.Decimal via sqlalchemy

Tue Dec 16 09:27:03 UTC 2025

** Also affects: cloud-archive/dalmatian
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/antelope
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/caracal
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/yoga
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/flamingo
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/gazpacho
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/epoxy
   Importance: Undecided
       Status: New

** Also affects: cloud-archive/ussuri
   Importance: Undecided
       Status: New

** Changed in: cloud-archive/dalmatian
       Status: New => Fix Released

** Changed in: cloud-archive/epoxy
       Status: New => Fix Released

** Changed in: cloud-archive/flamingo
       Status: New => Fix Released

** Changed in: cloud-archive/gazpacho
       Status: New => Fix Released

** Changed in: cloud-archive/caracal
       Status: New => In Progress

** Changed in: cloud-archive/caracal
     Assignee: (unassigned) => Seyeong Kim (seyeongkim)

** Also affects: cloud-archive/bobcat
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/2108994

Title:
  apscheduler retrieving decimal.Decimal via sqlalchemy

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive antelope series:
  New
Status in Ubuntu Cloud Archive bobcat series:
  New
Status in Ubuntu Cloud Archive caracal series:
  In Progress
Status in Ubuntu Cloud Archive dalmatian series:
  Fix Released
Status in Ubuntu Cloud Archive epoxy series:
  Fix Released
Status in Ubuntu Cloud Archive flamingo series:
  Fix Released
Status in Ubuntu Cloud Archive gazpacho series:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  New
Status in Ubuntu Cloud Archive yoga series:
  New
Status in watcher package in Ubuntu:
  Fix Released
Status in watcher source package in Focal:
  New
Status in watcher source package in Jammy:
  New
Status in watcher source package in Noble:
  In Progress
Status in watcher source package in Plucky:
  Fix Released
Status in watcher source package in Questing:
  Fix Released
Status in watcher source package in Resolute:
  Fix Released

Bug description:
  [Impact]
  apscheduler’s get_next_run_time() function expects the value to be of type float.
  However, SQLAlchemy’s scalar() may return a decimal.Decimal instead.
  In that case, apscheduler raises a TypeError.

  The issue is addressed by the following patch in Watcher, which explicitly handles the type conversion.
  This fix was merged starting with the 2024.2 release, so releases prior to 2024.1 do not include it.

  From e24cf31b608e92d9efe377662fba63887860c0c9 Mon Sep 17 00:00:00 2001
  From: Takashi Kajinami <kajinamit at oss.nttdata.com>
  Date: Wed, 8 May 2024 15:24:52 +0900
  Subject: [PATCH] SQLAlchemy 2.0: Omnibus fixes patch

  This was originally five patches, but they are all needed to pass
  any of the test jobs now, so they have been squashed into one:

  Co-Authored-By: Dan Smith (dms at danplanet.com)

  First:

  The autoload argument was removed[1] in SQLAlchemy and only
  the autoload_with argument should be passed.

  The autoload argument is set according to the autoload_with argument
  automatically even in SQLAlchemy 1.x[2] so is not at all needed.

  [1] https://github.com/sqlalchemy/sqlalchemy/commit/c932123bacad9bf047d160b85e3f95d396c513ae
  [2] https://github.com/sqlalchemy/sqlalchemy/commit/ad8f921e969b6f735dc8e08d882c961dde78f2b1

  Second:

  Remove _warn_on_bytestring for newer SA, AFAICT, this flag has been
  removed from SQLAlchemy and that is why watcher-db-manage fails to
  initialize the DB for me on jammy. This migration was passing the
  default value (=False) anyway, so I assume this is the right "fix".

  Third:

  Fix joinedload passing string attribute names

  Fourth:

  Fix engine.select pattern to use begin() per the migration guide.

  Fifth:

  Override the apscheduler get_next_run_time() which appears to be
  trivially not compatible with SQLAlchemy 2.0 because of a return type
  from scalar().

  Change-Id: I000e5e78f97f82ed4ea64d42f1c38354c3252e08

  [Test Case]
  1. deploy openstack env with watcher. ( based on juju )
  2. openstack optimize audit create   -t CONTINUOUS   --interval 60   --goal dummy   --strategy dummy   --name test-audit
  3. juju ssh watcher/0 -- sudo systemctl restart watcher-decision-engine
  4. You may see error msg like below. 
  - Dec 16 06:15:45 node-15 watcher-decision-engine[88884]: TypeError: 'decimal.Decimal' object cannot be interpreted as an integer

  [Where problems could occur]
  TBD 
  - tested Noble caracal, but need to test older versions.

  [Others]

  =original description=

  Kolla-Ansible: 2023.2
  Ubuntu 22.04.5 LTS

  I have a fresh installation in a three-node testing environment with
  Watcher enabled (deployed via Kolla Ansible).

  When creating a continuous audit, the watcher_engine container logs
  the following traceback:

  Exception in thread APScheduler:
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/var/lib/kolla/venv/lib/python3.10/site-packages/apscheduler/schedulers/blocking.py", line 32, in _main_loop
      wait_seconds = self._process_jobs()
    File "/var/lib/kolla/venv/lib/python3.10/site-packages/apscheduler/schedulers/base.py", line 1005, in _process_jobs
      jobstore_next_run_time = jobstore.get_next_run_time()
    File "/var/lib/kolla/venv/lib/python3.10/site-packages/apscheduler/jobstores/sqlalchemy.py", line 86, in get_next_run_time
      return utc_timestamp_to_datetime(next_run_time)
    File "/var/lib/kolla/venv/lib/python3.10/site-packages/apscheduler/util.py", line 179, in utc_timestamp_to_datetime
      return datetime.fromtimestamp(timestamp, utc)
  TypeError: 'decimal.Decimal' object cannot be interpreted as an integer

  Source:
  https://github.com/agronholm/apscheduler/blob/be291699755e58ff398f90b5e71bff1e163df1db/apscheduler/jobstores/sqlalchemy.py#L85

  For some reason, this line retrieves a decimal.Decimal, even though
  there’s no explicit configuration for float columns to default to
  asdecimal=False.

  This behavior aligns with the note by danms in the following commit:
  https://opendev.org/openstack/watcher/commit/d6f169197efc5b4f6c8a2e6bc38177b0641ca05c

  But danms assumes SQLAlchemy 2.0 stuff, which definetly isn't in my
  case.

  I also tested minimal Python scripts within the container to verify
  whether the issue is related to the versions of APScheduler (3.10.1)
  or SQLAlchemy (1.4.41), but I was unable to reproduce the problem. In
  those tests, float values were correctly returned from float-type
  columns.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/2108994/+subscriptions