.. _Business Intelligence: ===================== Business Intelligence ===================== In order to better understand client behaviour we've build a business intelligence analytics system. The key components are as follows: - Mongodb database - MEG celery tasks to push aggregated data to the mongodb database - Mongodb Charts reports which query the mongodb database Mongodb database ---------------- The mongodb database is hosted in Azure, but can be managed via the `Mongodb Atlas website `_. IP restriction is in place to only allow connections to the database from our production servers. The process of adding a new index should be managed via the Mongodb Atlas website. Development Usage Guide ----------------------- For development there is a docker compose setup to simulate the production mongodb database. Run the following command to launch the MEG dev container and the mongodb containers: .. code-block:: bash BI_AGGREGATE=1 docker compose --profile bi up Once the containers are up and running you can view the mongo database `here `_ You will need to enter the username (`admin`) and password (`password`) to login to mongo express. Scheduled Tasks ================= The following tasks run on a schedule on each server to aggregate data to the Mongodb database. The tasks only function if the environment variable :envvar:``BI_AGGREGATE`` is enabled. These tasks can be run manually via the `periodic tasks `_ django admin page. .. note:: Some tasks wipe the existing data in the Mongodb collection while others append to the existing data. Tasks that append to existing data usually add data for the prior month to the existing collection. When manually running these tasks it's advisable to first wipe the data for the collection in Mongodb. You can do this in `Mongodb `_. .. automodule:: client_management.tasks :members: aggregate_monthly_submissions, aggregate_monthly_sessions, aggregate_quarterly_form_churn, sync_institutions, sync_security_risk_clients, sync_forms, sync_top_users