Background Job Management System.
With a web interface, users can utilize tools that
involve running scripts for tasks such as report data,
file conversion and cleaning records in database tables.
The target is to enhance the user experience by allowing
users to add these activities from a web page panel,
set their options, and run them without having to wait
for the job to complete.
Users should be able to leave
the page and return later, or receive an email notification
when their jobs are finished.
Job Queue System
A job queue to manage and run user-submitted tasks in the background.Ensure the system can handle various types of jobs, including export in various formats, file conversion and database operations.
Web Interface (User-friendly)
View a list of available activities.Configure options for each activity.
Submit the job to be processed.
Allow users to view the status of their submitted jobs.
Background Processing
Background worker that processes jobs from the queue.Check for new jobs.
Execute the job.
Register key activities, such as start time, end time, and significant steps in more complex jobs.
Notification System
Email notification system to alert users when their jobs are completed.Users can receive notifications without needing to stay on the web page.
Technical Considerations
Consider libraries or frameworks for job queues (e.g., Celery for Python, Bull for Node.js).Explore message brokers like Redis or RabbitMQ for managing the job queue.
Ensure the web interface is developed using a suitable framework (e.g., Flask or Django for Python, Express for Node.js).
Implement an email service (e.g., Nodemailer for Node.js, Flask-Mail for Flask, Django's built-in email system).