Skip to content

Instantly share code, notes, and snippets.

@sehmaschine
Created March 2, 2013 13:54
Show Gist options
  • Select an option

  • Save sehmaschine/5071098 to your computer and use it in GitHub Desktop.

Select an option

Save sehmaschine/5071098 to your computer and use it in GitHub Desktop.
celery update_state issue
from tasks import full_export_task
def export_year_zip(user):
full_export_task.update_state(state="PROGRESS", meta={})
task()
def full_export_task(user, task_id=None):
from export.views.user_full_export import export_year_zip
result = export_year_zip(user)
return result
@sehmaschine
Copy link
Copy Markdown
Author

The above code works fine. However, I'm not sure it is supposed to work like this ... how does celery know which task to actually update? I can run several tasks and the PROGESS–state is updated nicely. But I don't use a task_id to identify the task which should be updated.

BTW: Using celery 2.5.5

@honza
Copy link
Copy Markdown

honza commented Mar 2, 2013

I think the idea of the custom task state is to notify any applications that are aware of this running of the changing state. For example, suppose you have a video transcoding job that you run in a task. This is done in chunks because the video is very large. Before you start processing the video, you know you have 100 chunks to process. The update_state method will allow you to update the task's state (i.e. progress in the video example) as the task is running. This means that you can say, "If anyone cares, I have just finished the first chunk of 100. I'll let you know when I'm done with the next one."

The reason why there is no reference to task_id is because it refers to the current task.

Does that make sense?

@sehmaschine
Copy link
Copy Markdown
Author

The above code is just an example. export_year_zip is actually much longer and does exactly what you've described: telling the user how much of the export has already been processed with a long–running task.

The problem I'm having is this: You say that it refers to the "current task". But how does celery know what the "current task" is when I'm having 4 tasks running synchronously? I just want to prevent that all 4 tasks suddenly have the same state. And before I push this to the live instance I wanted to make sure that this code is supposed to work (although the tests are running fine).

@honza
Copy link
Copy Markdown

honza commented Mar 2, 2013

I think the problem is that you're calling update_state from outside the @task-decorated function. Not really sure, sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment