Skip to content

Multiple instances of shceduler problem #51

Closed
@shr-chauhan

Description

@shr-chauhan

I had a doubt, about using flask-apscheduler in a web app
When used in any forking web server (gunicorn uwsgi, etc...), processes get forked, thus apacheduler gets instantiated (and possibly started) in each forked worker process. This leads to the "many jobs running" issue?
In production multiple instances of the app gets created, it will initiate that many scheduler instances also right? so all those instances will try executing the job is it?
Am I wrong here?

Activity

viniciuschiele

viniciuschiele commented on Aug 16, 2017

@viniciuschiele
Owner

yeah, your assumption is right, it will start one scheduler for each forked process and it will run the same job multiple times.
If you are running this service on multiple servers, you could keep only one active and the others as idle,
we achieved it using uwsgi legion.

everhopingandwaiting

everhopingandwaiting commented on Sep 1, 2017

@everhopingandwaiting
            if os.environ.get("scheduler_lock") == "1":
                scheduler.start()
                logging.debug("Schedular Started ,-------------")
                os.environ["scheduler_lock"] = os.environ.get("scheduler_lock") + "1"

before start the program ,you had better to export scheduler_lock="1"

shr-chauhan

shr-chauhan commented on Sep 1, 2017

@shr-chauhan
Author

Sorry for the late update
@everhopingandwaiting yea that is one way of doing it, but I am doing it differently
Running scheduler as a rpc server outside the app, so its a different process now....so whenever there is a scheduling request, it gets connected to the rpc and process the request there....

@viniciuschiele closing this

everhopingandwaiting

everhopingandwaiting commented on Sep 2, 2017

@everhopingandwaiting
chenxuewen

chenxuewen commented on Aug 13, 2020

@chenxuewen

Why did you add this code? Unable to dynamically modify timing tasks?

import atexit
import fcntl
from apscheduler.events import EVENT_JOB_ERROR, EVENT_JOB_MISSED, EVENT_JOB_EXECUTED
def scheduler_init(app):

f = open("scheduler.lock", "wb")
try:
    fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)


    scheduler.add_listener(job_listener, EVENT_JOB_ERROR | \
                       EVENT_JOB_MISSED | \
                       EVENT_JOB_EXECUTED)
    scheduler.init_app(app)
    scheduler.start()
except:
    pass
def unlock():
    fcntl.flock(f, fcntl.LOCK_UN)
    f.close()
atexit.register(unlock)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @viniciuschiele@everhopingandwaiting@shr-chauhan@chenxuewen

        Issue actions

          Multiple instances of shceduler problem · Issue #51 · viniciuschiele/flask-apscheduler