This is part one of our series about providing initial data in Django web projects. Here is part two. This post is targeted at developers, and especially developers interested in Wagtail, Python and Django. You probably need some basic Django knowledge to make the most out of reading this article.
Here at Codista we build a lot of websites and web applications. The process of setting up a project is always quite similar: you start a project using a cookiecutter or via django-admin startproject mysite.
Followed by a few manual steps like:
- Setting up your database
- Creating and running database migrations
- Creating an admin user via
python manage.py createsuperuser - Setting up some initial project data or providing some data fixtures
After that you start developing features - your first database model, creating and running database migrations and then you hook up the first view to check out if the basic project setup is working properly.
Nothing fancy until this point. Now you are probably not developing this project alone, but you have a team providing different skills to the project. Probably a frontend developer who is mastering HTML, CSS and JS, some more backend developers and maybe even a data scientist.
The problem
At this point in the project you might ask yourself how to onboard your team members as fast and consistent as possible, without having to force them to do all these manual steps over and over again.
Since we are developers we like automating away repeating tasks, as it makes our lives easier. So we came up with an automated solution that we call the TOTAL SETUP.
Onboarding team members is just one use case for it, it can also be used to achieve the following tasks:
- resetting the local environment (for example after you reset your database migrations), with initial data available afterwards
- setting up the initial production data when we do the first deployment
- setting up a staging server with initial data
- resetting your project to a clean state after you messed up with the data or did not touch the project for a long time
That's why this approach can save you and your team a lot of time and headaches.
The Total Setup Approach
So let’s dive in. At the core of our approach are three management commands:
total_resettotal_setupsetup_page_tree
The total_reset command is the main command which will be executed by team members to setup their local environment. It has the following goals:
- Create or recreate the database
- Run the database migrations
- Call all other management commands to setup your project data
The code below should give you an idea how the total_reset is structured. You can find the final snippet in our GitHub repository.
class Command(BaseCommand):
"""DEV ONLY: Dumps the entire DB and sets up everything anew."""
requires_system_checks = False
def _terminate_db_connections(self, database):
"""Terminates the database connections to be able to drop the database"""
...
def _create_db(self, database):
"""Sets up the database"""
...
def _drop_db(self, database):
"""Drops the database"""
...
def _create_or_recreate_db(self, database):
"""creates or recreates the database"""
...
def handle(self, *args, **options):
"""entry point"""
...
Using the total_reset command helps to automate the repetitive task of recreating a database. But there is more to it - it calls a few other management tasks to execute further setup tasks. For example it executes the migrations:
call_command("migrate", verbosity=verbosity)
It will also run the total_setup command, which is responsible for setting up initial project data & settings. It will:
- Create the admin users
- Setup the correct domain if you are using Django’s Site Framework
- Setup the wagtail domain if you are using Wagtail
- Setup all needed project data & provide data fixtures
Here's how this command looks. You can find the final snippet here.
class Command(BaseCommand):
"""Sets up initial project data & settings. Also in production!"""
def _set_domain(self):
"""Sets the django and wagtail domains. Across all environments."""
current_site = Site.objects.get_current()
if settings.DEBUG:
current_site.domain = "localhost:3000"
elif getattr(settings, "STAGING", False):
current_site.domain = "test.codista.com"
else:
current_site.domain = "www.codista.com"
current_site.save()
if PROJECT_USES_CMS:
wagtail_site = WagtailSite.objects.get()
wagtail_site.hostname = current_site.domain
wagtail_site.save()
def setup_production(self):
"""PRODUCTION ONLY STUFF."""
self._set_domain()
call_command("create_project_users", verbosity=self.verbosity)
def setup_development(self):
"""DEVELOPMENT ONLY STUFF."""
self.setup_production()
def handle(self, *args, **options):
"""entry point"""
self.verbosity = options["verbosity"]
if not settings.DEBUG:
if self.verbosity > 0:
self.stdout.write("Setting up production defaults...")
self.setup_production()
return
if self.verbosity > 0:
self.stdout.write("Setting up sensible development defaults...")
self.setup_development()
You might have noticed the create_project_users command. This command is responsible for setting up the admin users for your team. You can find the full command on Github.
Takeaways
If you made it this far, you have already automated quite a bunch of repetitive tasks that would otherwise have been required to run manually. Using this approach we reached the following status:
- Automate some simple, but as a whole quite time-consuming tasks
- Creating and recreating the database
- Creating your user and your team members as Django admins
- Running the database migrations
- Provide a process to consistently setup initial project data where you can hook into various management commands to setup additional project specific initial data & fixtures
- Achieve consistency and predictability in setting up initial project data
In the end, all that's left is to execute ./manage.py total_reset to set up a consistent initial data set across your entire team.
If you want to learn more about providing initial data in Django / Wagtail projects, continue reading Part Two of this blog post series.
