Migrations

Migrations are Django's way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They act as a version control system for your database, allowing you to develop schema changes repeatedly and share them across different environments (development, staging, production).

While migrations are mostly automatic, understanding the underlying commands and operations is essential for handling complex changes and data migrations.

Core Commands

These are the primary utilities for interacting with migrations:

• makemigrations: Scans your models and compares them to the current migration state. It packages these changes into new migration files (Ex: 0001_initial.py).
• migrate: Synchronizes the database state with your current set of models and migrations. It applies or un-applies migrations to the database.
• sqlmigrate: Displays the raw SQL statements that a migration will run. Useful for verifying what Django is about to do to your database.
• showmigrations: Lists all migrations for the project and their current status (applied or unapplied).

The Migration Workflow

The standard workflow for making schema changes is:

1. Modify Models: Make changes to your models.py (Ex: add a field, remove a model).
2. Create Migrations: Run python manage.py makemigrations.
3. Apply Migrations: Run python manage.py migrate.
4. Commit: Commit both the updated models.py and the generated migration files to version control.

NOTE

Django generates migrations for any change to your models or fields, even those that don't alter the database (like validators or help text). This ensures Django can accurately reconstruct historical models for use in future data migrations.

App-Specific Migrations

You can restrict operations to a specific app by providing its app label as an argument:

You can restrict migrations for a single app by ...

python manage.py makemigrations app_name
python manage.py migrate app_name

Restricting migrations to a single app is a best-effort promise, not a guarantee. Django will automatically apply migrations from other apps if they are required to satisfy dependencies (Ex: a ForeignKey pointing to a model in another app).

Naming Migrations

To generate a descriptive filename for your migrations, use --name flag. Example:

python manage.py makemigrations --name changed_my_model your_app_label

Migration Files

NOTE

Migration files are stored in the migrations/ directory of each app. It is possible to override this location globally or on a per-app basis using the MIGRATION_MODULES setting.

Migration files are standard Python modules stored in your app's migrations/ directory. Each file defines a Migration class (a subclass of django.db.migrations.Migration) that instructs Django on how to apply changes. The class relies on two primary attributes:

• dependencies: A list of previous migrations that must be applied before this one.
• operations: a list of Operation objects that define the specific changes to be performed.

from django.db import migrations, models


class Migration(migrations.Migration):
    dependencies = [("myapp", "0001_initial")]

    operations = [
        migrations.DeleteModel("Tribble"),
        migrations.AddField("Author", "rating", models.IntegerField(default=0)),
    ]

You rarely need to edit migration files by hand, but manual edits are sometimes necessary for complex operations or conflict resolution.

Generating an Empty Migration

To write a custom migration, start by generating a blank file as shown below. Django will handle the boilerplate (timestamps, names, and dependencies) for you, leaving you with an empty operations list to fill.

python manage.py makemigrations --empty app_name

Migration Operations

As mentioned, Migration files consist of Operation objects that declaratively describe what actions to take.

Schema Operations

These operations alter the database structure:

• CreateModel: Creates a new table.
• DeleteModel: Drops a table.
• RenameModel: Renames a table.
• AddField: Adds a column.
• RemoveField: Removes a column.
• AlterField: Modifies a column's definition (type, null status, etc.).
• RenameField: Renames a column.
• AddIndex / RemoveIndex: Manages database indexes.
• AddConstraint / RemoveConstraint: Manages database constraints.

Special Operations

These allow for data manipulation and raw SQL execution:

• RunPython: Executes custom Python code in a historical context. Commonly used for Data Migrations (covered below).
  • Arguments: code (forward function), reverse_code (rollback function).
• RunSQL: Executes arbitrary SQL on the database.
  • Arguments: sql (forward SQL), reverse_sql (rollback SQL).
• SeparateDatabaseAndState: A specialized operation that allows applying operations to the database (SQL) and the internal project state separately. Useful for complex operations like changing a ManyToManyField to use a "through" model without data loss.

For a complete list of Migration Operations, refer to the official documentation.

The django_migrations Table

Django tracks the state of your database using a standard database table named django_migrations. This table acts as the "history book" for your schema, recording exactly which migration files have been applied to the current database.

Table Structure

The table is simple and typically contains the following columns:

Column	Description
id	Primary key (auto-incrementing integer).
app	The name of the Django app (Ex: auth, users).
name	The exact name of the migration file (Ex: 0001_initial).
applied	A timestamp indicating when the migration was executed.

How It Works

When you run python manage.py migrate:

1. Django looks at the migration files in your code (migrations/ folders).
2. It compares them against the records in the django_migrations table.
3. If a file exists in code but not in the table: Django runs the migration and adds a new row to the table.
4. If a file exists in the table: Django skips it, assuming the changes are already present in the database.

Handling Conflicts

IMPORTANT

Migrations should be treated as part of your codebase and must be committed to version control.

Duplicate migration numbers (Ex: two 0004 files) caused by parallel development are rarely an issue, as Django relies on internal dependencies rather than filenames.

When conflicts occur, Django will detect them and:

1. Prompt to automatically linearize the migrations if safe.
2. Ask for manual intervention if the dependency order is ambiguous, requiring you to edit the dependencies list in the migration file.

Handling Existing Schemas

If you are adding migrations to an app that already has populated tables in the database (Ex: converting a legacy project to use migrations), standard migrations will fail because the tables already exist. To solve this, use the --fake-initial flag:

# 1. Create the initial migration file based on current models
python manage.py makemigrations

# 2. Apply the migration, skipping SQL creation if tables exist
python manage.py migrate --fake-initial

When you run migrate --fake-initial, Django detects if the migration is the "initial" one for an app and checks the database for the corresponding tables or columns. If it finds that they already exist, it skips the SQL execution (such as CREATE TABLE) and simply marks the migration as "applied". Otherwise, it proceeds to apply the migration normally.

--fake-initial vs --fake

Both flags instruct Django to record a migration as "applied" without executing its SQL, but they differ significantly in their logic and safety.

• --fake: This is unconditional. It tells Django to mark the specific migration(s) as applied, regardless of the database state. You use this when you know the database schema is already up-to-date, but the django_migrations table does not reflect it. This simply add row to the django_migrations table without running the SQL.
• --fake-initial: This is conditional and safer. It only applies to migrations marked as initial=True. It checks if the table (or columns) already exist in the database:
  • If found: It skips the SQL (acts like --fake).
  • If missing: It runs the SQL normally.

Reversing Migrations

WARNING

Reversing a migration usually involves dropping tables or columns. Any data stored in those structures will be permanently lost. Always back up your data before reversing migrations in production.

To undo changes or roll back your database schema to a previous state, you simply pass the number of the migration you want to revert to.

For example, to revert 0003 and return the database to the state of 0002:

python manage.py migrate myapp 0002

Reversing All Migrations

To undo all migrations for an app (effectively resetting it to a pre-migration state), use the special name zero:

python manage.py migrate myapp zero

This removes all rows from the django_migrations table for myapp app and reverses the schema changes in the database.

Irreversible Migrations

Some migrations are strictly one-way. This occurs when a change destroys data (like deleting a column) or when custom code (using RunPython) lacks "undo" instructions. If you try to reverse these, Django raises an IrreversibleError because it cannot restore the lost data.

To avoid this in custom migrations, you must provide reverse_code. If the reverse action should do nothing, you can use RunPython.noop.

migrations.RunPython(forward_func, reverse_code=RunPython.noop)

Squashing Migrations

Projects often accumulate hundreds of migration files over time. You can use squashing to reduce these to a manageable number.

Squashing is the act of reducing an existing set of many migrations down to one (or sometimes a few) migration which still represent the same changes.

# Squash migrations 0001 through 0004 into a single file
python manage.py squashmigrations myapp 0004

Post Migration

Once you’ve squashed your migrations, do not delete the old migration files immediately. You should first commit the new squashed migration alongside the old ones. Once you have confirmed that all environments (production, staging, other developers) have applied the new history, you can then:

• Delete the old migration files.
• Update all migrations that depend on the deleted migrations to depend on the squashed migration instead.
• Remove the replaces attribute in the Migration class of the squashed migration.

Named Migrations

To add a custom name to your squashed migration, use:

python manage.py squashmigrations myapp 0004 --squashed-name=my_custom_name

Writing Advanced Migrations

Migrations aren't just for schema changes, they can also modify the data itself. These are called "Data Migrations" and are commonly used to populate new fields, normalize data, or backfill values.

Data Migrations

To modify data alongside schema changes (Ex: moving data from one column to another), use RunPython.

You must use apps.get_model('app_name', 'ModelName') inside the RunPython function to access a model rather than importing it directly. This ensures you are interacting with the version of the model that exists at that point in the migration history.

from django.db import migrations

def combine_names(apps, schema_editor):
    # We can't import the Person model directly as it may be a newer
    # version than this migration expects. We use the historical version.
    Person = apps.get_model("yourappname", "Person")
    for person in Person.objects.all():
        person.name = f"{person.first_name} {person.last_name}"
        person.save()

class Migration(migrations.Migration):
    dependencies = [("yourappname", "0001_initial")]

    operations = [
        migrations.RunPython(combine_names),
    ]

Multiple Databases

If your project uses multiple databases, check the database alias to ensure the migration runs on the correct one.

from django.db import migrations

def forwards(apps, schema_editor):
    # Ensure this runs only on the default database
    if schema_editor.connection.alias != "default":
        return
    # Your migration code goes here

class Migration(migrations.Migration):
    dependencies = [("yourappname", "0001_initial")]
    operations = [
        migrations.RunPython(forwards),
    ]

Handling Unique Fields

Adding a unique, non-nullable field to a table with existing rows causes errors because existing rows will violate the uniqueness constraint. To address this:

1. Add the field as nullable (null=True).
2. Create a data migration (RunPython) to populate the field with unique values (Ex: UUIDs).
3. Alter the field to be non-nullable and unique (null=False, unique=True).

The following example populates UUIDs for an attribute in MyModel model:

import uuid
from django.db import migrations

def gen_uuid(apps, schema_editor):
    MyModel = apps.get_model("myapp", "MyModel")
    # Update all rows with a new UUID
    for row in MyModel.objects.all():
        row.uuid = uuid.uuid4()
        row.save(update_fields=["uuid"])

class Migration(migrations.Migration):
    dependencies = [("myapp", "0004_add_uuid_field")]
    operations = [
        # omit reverse_code=... if you don't want the migration to be reversible.
        migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
    ]

Non-Atomic Migrations

By default, on databases that support DDL transactions (PostgreSQL, SQLite), migrations run inside a single transaction. For massive data updates, this can lock tables for too long.

To avoid this, set atomic = False on the Migration class. You can then process data in smaller batches or manually control transactions.

Controlling Order

• Dependencies: Migrations automatically declare dependencies on previous migrations. This is the standard way to ensure order within an app or across apps.
• run_before: You can explicitly force a migration to run before another migration in a different app using the run_before attribute.

class Migration(migrations.Migration):
    dependencies = [("myapp", "0001_initial")]
    
    # This migration must finish before 'third_party_app' starts its 0001 migration
    run_before = [
        ("third_party_app", "0001_do_awesome"),
    ]

Complex Schema Changes (Zero Downtime)

Some schema changes require refactoring without losing data or taking the application offline. A common example is converting a ManyToManyField into one that uses an explicit "through" model.

In these cases, Django’s SeparateDatabaseAndState operation lets you change the database schema and Django’s model state independently. This allows you to reuse the existing join table instead of dropping and recreating it.

Suppose you had a Book model with a ManyToManyField linking to Author model and Django created an implicit join table named core_book_authors. You now want to introduce a custom "through" model called AuthorBook.

from django.db import migrations, models

class Migration(migrations.Migration):
    operations = [
        migrations.SeparateDatabaseAndState(
            database_operations=[
                # Rename the existing many-to-many table instead of dropping it
                migrations.RunSQL(
                    sql="ALTER TABLE core_book_authors RENAME TO core_authorbook",
                    reverse_sql="ALTER TABLE core_authorbook RENAME TO core_book_authors",
                ),
            ],
            state_operations=[
                # Update Django’s model state to reflect the new through model
                migrations.CreateModel(
                    name="AuthorBook",
                    fields=[
                        ("id", models.BigAutoField(primary_key=True, serialize=False)),
                        ("author", models.ForeignKey(on_delete=models.CASCADE, to="core.author")),
                        ("book", models.ForeignKey(on_delete=models.CASCADE, to="core.book")),
                    ],
                    options={
                        "db_table": "core_authorbook",
                        "unique_together": {("author", "book")},
                    },
                ),
            ],
        ),
    ]

Serializing Values

Migrations are Python files that describe your database changes. To create them, Django must serialize (convert) the current state of your Python objects into text code.

Django automatically handles standard Python types like integers, lists, and strings. However, it cannot natively serialize custom class instances or lambdas.

Custom Serializers

To serialize custom classes, you can add a deconstruct() method that returns class's import path and constructor arguments. For simpler classes where the arguments are already serializable, use the @deconstructible decorator. This ensures that the arguments passed to your class are preserved in the migration file.

from django.utils.deconstruct import deconstructible
from django.db import models

@deconstructible
class SizeValidator:
    def __init__(self, max_size):
        self.max_size = max_size

    def __eq__(self, other):
        return self.max_size == other.max_size

class MyModel(models.Model):
    # Django can now serialize this validator into the migration file
    name = models.CharField(max_length=100, validators=[SizeValidator(10)])

Backend Support

• PostgreSQL: The most capable backend. Supports transactional DDL (you can roll back schema changes).
• MySQL: Does not support transactions around schema changes. If a migration fails halfway, you must manually fix the database state.
• SQLite: Has limited schema alteration support. Django emulates many operations (like dropping a column) by creating a new table, copying data, and swapping the tables. This can be slow and brittle.

SchemaEditor

The SchemaEditor is the database abstraction layer that translates Django's migration operations into database-specific SQL.

• It is typically used internally by Django and rarely accessed directly, except when writing custom migration operations.
• It exposes methods like create_model, add_field, delete_model and execute (for raw SQL).
• It is accessible as a context manager (see example below) to ensure database schema changes are executed safely and correctly.

with connection.schema_editor() as schema_editor:
    schema_editor.delete_model(MyModel)