Python News: What's New From September 2023

Modular: The Company Behind Mojo

Founded just last year, Modular is a promising startup company whose mission is to increase the pace of innovation in the field of artificial intelligence (AI):

Our mission is to have real, positive impact in the world by reinventing the way AI technology is developed and deployed into production with a next-generation developer platform. (Source)

The current AI platforms suffer from fragmented infrastructure with a high degree of complexity, making the deployment of machine learning workloads difficult. The company aims to address these challenges by offering a single platform to handle models from various AI frameworks.

The CEO and one of the co-founders of Modular is none other than Chris Lattner, renowned for inventing the LLVM compiler over two decades ago and, more recently, the Swift programming language during his time at Apple. Now, he’s using his expertise in compiler design to build Mojo, a brand-new programming language based on Python, which can leverage the heterogeneous multi-core hardware architectures of modern computers to make performance gains.

Mojo: A Superset of Python

Because Python powers the overwhelming majority of AI models, Mojo was designed as a Python superset to be compatible with existing machine learning libraries, frameworks, and codebases. This lowers the entry barrier while making developers more productive. At the same time, Mojo combines the elegance of the familiar Python syntax with the speed of compiled languages like C and C++. In some cases, it can run up to 68,000 times faster than Python!

During their product launch keynote earlier this year, Modular sparked quite a lot of excitement in the Python and data science communities by revealing Mojo to the world. It was initially only available through a limited preview in an online playground. Four months later, the Mojo compiler is available for download, making offline development possible.

Today, anyone can sign up for a Modular account and download the Software Development Kit (SDK) for Mojo, which includes a compiler for Linux.

Mojo Compiler

While the Mojo SDK is only available for Ubuntu Linux at the moment, other platforms will gain native support in future releases. In the meantime, Modular recommends running the SDK on macOS and Windows through a Docker container, a remote virtual machine in the cloud, or GitHub Codespaces.

$ git clone git@github.com:modularml/mojo.git
$ cd mojo/examples/docker/
$ chmod +x build-image.sh
$ ./build-image.sh --auth-key <YOUR_AUTH_TOKEN>

$ docker images
REPOSITORY                        TAG       IMAGE ID       ...   SIZE
modular/mojo-v0.3-20232809-1947   latest    1c729f7db052   ...   2.58GB

$ docker run --rm -it modular/mojo-v0.3-20232809-1947 mojo
Welcome to Mojo! 🔥

Expressions are delimited by a blank line.
Type `:quit` to exit the REPL and `:mojo help` for further assistance.

  1>

$ docker run --rm -p 8888:8888 -v $PWD:/shared modular/mojo-v0.3-20232809-1947
$ open http://localhost:8888/lab

The first step to getting the Mojo SDK onto a local machine is to download and install the Modular CLI, which provides the modular command in the terminal. Among other things, it allows you to install or update the Mojo SDK like so:

$ modular install mojo
# Found release for https://packages.modular.com/mojo @ 0.3.1
# Installing to /home/realpython/.modular/pkg/packages.modular.com_mojo
# Downloading artifacts. Please wait...
# Downloads complete, setting configs...
# Configs complete, running post-install hooks...

(...)

🔥 Mojo installed! 🔥

$ modular update mojo
# No updates available for this target.

Now you have the Mojo CLI, comprising a single mojo command. To confirm that you’ve installed the Mojo SDK correctly, you can check the current Mojo version:

$ mojo --version
mojo 0.3.1 (a3eed7c8)

The mojo command comes with subcommands that integrate several tools, including a code formatter, documentation generator, compiler, and a REPL (Read-Eval-Print Loop) for Mojo.

Mojo Interpreter

The bare mojo command is equivalent to typing mojo repl at the command prompt, which launches the Mojo REPL—an interactive shell for Mojo, resembling the Python REPL:

$ mojo
Welcome to Mojo! 🔥

Expressions are delimited by a blank line.
Type `:quit` to exit the REPL and `:mojo help` for further assistance.

  1> print(
  2.   "Hello, World!"
  3. )
  4. print("Bye!")
  5.
Hello, World!
Bye!

Unlike the standard Python REPL, the Mojo REPL allows you to type multiline expressions or even several expressions at a time, and it expects a blank line to indicate where they end. Perhaps the developers decided to include such a behavior because Mojo compiles the code in the background, which takes extra time. Without this behavior, you might perceive the experience of running code in the Mojo REPL as slow.

Mojo Source Files

While the Mojo REPL allows you to execute code at the top level, Mojo source files don’t. Expressions are only allowed inside nested scopes, and programs must contain the main() function as an entry point unless they’re utility modules:

# hello.mojo

fn main():
    print("Hello, World!")

This is typical of compiled programming languages like C, C++, or Java, which also require an entry point. Note that the print() function in the code snippet above isn’t the same as the print() function in Python! In particular, these functions take different parameters.

To execute a Mojo source file, you can run it directly with the mojo command or the more explicit mojo run subcommand:

$ mojo hello.mojo
Hello, World!

$ mojo run hello.mojo
Hello, World!

In both cases, the source file gets compiled into machine code behind the scenes and discarded immediately afterward. Building a reusable binary file boils down to typing mojo build followed by the Mojo source file:

$ mojo build hello.mojo

$ ./hello
Hello, World!

$ du -h hello
28K hello

$ file hello
hello: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
⮑ dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,
⮑ BuildID[sha1]=ca7b91bfd99556679d79a67852d684ccb3cba4c2,
⮑ for GNU/Linux 3.2.0, not stripped

This generates a binary executable, which is relatively small when compared to executables generated by, say, Go. At the same time, the binary file produced by the Mojo compiler enjoys broad portability across Linux distributions that meet the minimum kernel version and target the same CPU architecture. As a result, Mojo programs are much more straightforward to distribute than Python programs.

Mojo and Python Interoperability

Mojo is currently in its infancy, making it a work in progress, so it’s far from being a complete superset of Python. That said, there are ways to call out existing Python code and combine it with Mojo.

Either in the Mojo REPL or a notebook’s cell that uses the Mojo kernel for Jupyter, you can use the %%python cell magic to interpret Python code:

$ mojo
Welcome to Mojo! 🔥

Expressions are delimited by a blank line.
Type `:quit` to exit the REPL and `:mojo help` for further assistance.

  1> %%python
  2. import sys
  3. print(sys.version)
  4.
3.11.5 (main, Aug 30 2023, 17:24:45) [GCC 12.2.0]
(PythonObject) sys = {
  (PyObjectPtr) py_object = {
    (pointer<scalar<si8>>) value = 0x00007f4a9557aca0
  }
}

Mojo runs Python from a special virtual environment created at installation time. Currently, the installer picks one of the Python interpreters found on the machine without asking the user.

To run Python in a Mojo source file, you need to import the Python interface:

# hello.mojo

from python import Python

fn main() raises:
    let sys = Python.import_module("sys")
    print(sys.version)

Later, it’s possible to access the values computed by Python in the subsequent Mojo code.

Check the official Mojo documentation to learn more about Python interoperability, the Mojo standard library, and more. You can also take a look at a few code examples in the official Mojo repository on GitHub.

Do you share the excitement around Mojo, or do you think it’s just temporary hype? Is Mojo going to replace Python? Tell us in the comments below!

Database-Computed Defaults

An interesting feature introduced in Django 5.0 is the ability to compute default model field values directly at the database level rather than through Django:

# models.py

from django.db import models
from django.db.models.functions import Now
from django.utils import timezone

class BlogPost(models.Model):
    title = models.CharField(max_length=60)
    created_by_django = models.DateTimeField(default=timezone.now)
    created_by_database = models.DateTimeField(db_default=Now())

Previously, you could use the default parameter in model fields to declare the default value in case one wasn’t provided at the time of object creation. Django would use that value at runtime before sending a query to the database. In contrast, the new db_default parameter alters the database schema generated by Django:

sqlite> .schema myapp_blogpost
CREATE TABLE IF NOT EXISTS "myapp_blogpost" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "created_by_django" datetime NOT NULL,
    "created_by_database" datetime DEFAULT (
        STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')
    ) NOT NULL
);

The table column named created_by_database in the example above specifies the DEFAULT constraint, unlike the previous column, which doesn’t. The extra constraint reflects the expression assigned to the db_default parameter in Python. Starting from Django 5.0, the web framework allows for the use of database functions, such as Now(), in such expressions.

Not only is this quicker to compute, but it also leads to better data consistency because there’s only one place with a complete set of constraints. Previously, it was possible to insert rows into the table while bypassing the model constraints defined in Django—for example, by executing a script.

Virtual and Persisted Columns

Another new feature related to databases that will make its way to Django 5.0 is the ability to define virtual columns through the web framework:

# models.py

from django.db import models
from django.db.models import F
from django.db.models.functions import Concat

class User(models.Model):
    first_name = models.CharField(max_length=100)
    last_name = models.CharField(max_length=100)
    full_name = models.GeneratedField(
        expression=Concat(F("first_name"), " ", F("last_name")),
        db_persist=False
    )

The new GeneratedField type defines a table column generated by the database, provided that the underlying database engine has such a capability. In this case, the database will compute the value of the full_name field from other fields.

When the db_persist parameter is set to False, the corresponding value will be computed on the fly without taking extra space on disk. On the other hand, querying the database will take longer due to the additional computation that’s necessary with each query.

Sometimes, it’s more desirable to compute the column’s value based on other columns when updating or inserting a new row into the table. Unlike virtual columns, a persistent column is stored on disk, consuming additional space. With Django 5.0, you can set the db_persist parameter’s value to True in order to define such a stored column:

# models.py

from django.db import models
from django.db.models import F
from django.db.models.functions import Concat

class User(models.Model):
    first_name = models.CharField(max_length=100)
    last_name = models.CharField(max_length=100)
    full_name = models.GeneratedField(
        expression=Concat(F("first_name"), " ", F("last_name")),
        db_persist=True
    )

As a result, selecting rows from the corresponding table will be as quick as if the generated column weren’t managed by the database. At the same time, you won’t be able to update virtual columns by hand.

Improved Model Field Choices

Continuing with the theme of Django models, the new version of the popular web framework will accept a few additional data types as the choices parameter of model fields. Traditionally, specifying a limited number of options to choose from in a Django model required defining a sequence of pairs:

# models.py

from django.db import models

class Person(models.Model):
    AGE_CHOICES = [
        ("M", "Minor"),
        ("A", "Adult"),
    ]
    age = models.CharField(max_length=1, choices=AGE_CHOICES)

That wasn’t particularly intuitive. Remembering which of the two values corresponds to a human-readable representation or the actual value stored in the database wasn’t always straightforward. Moreover, translating between the two could sometimes get clunky.

Django 5.0 will allow the use of a Python dictionary or any other mapping container type in addition to the classic iterable of tuples:

# models.py

from django.db import models

class Person(models.Model):
    AGE_CHOICES = {
        "M": "Minor",
        "A": "Adult",
    }
    age = models.CharField(max_length=1, choices=AGE_CHOICES)

The advantage of using a dictionary is that it simplifies the mapping of the value retrieved from the database into a descriptive label.

Another new type allowed in this context will be a callable object, such as a function:

# models.py

from django.db import models

def age_choices(age_types=("Minor", "Adult")):
    return {name[0]: name for name in age_types}

class Person(models.Model):
    age = models.CharField(max_length=1, choices=age_choices)

Calling a function to determine the choices may be suitable when the fixed set of values depends on some inputs supplied at runtime. Alternatively, it may be convenient to compute the choices dynamically—for example, by processing the contents of a large file.

Finally, you won’t need to access the .choices attribute of an enumeration type anymore starting from Django 5.0:

# models.py

from django.db import models

class Person(models.Model):
    class Age(models.TextChoices):
        MINOR = "M", "Minor"
        ADULT = "A", "Adult"
    age = models.CharField(max_length=1, choices=Age)

Previously, you would’ve had have to supply Age.choices instead of just Age in the last line above.

Reusable Form Field Groups

Django templates will gain a new way of rendering groups of elements related to a form field. Rather than laying out the individual components like the field label, HTML widget, help text, and error messages separately, you can now leverage the .as_field_group property:

<form action="{% url 'form_view' %}" method="post">
  {% csrf_token %}
  {{ form.name.as_field_group }}
  {{ form.value.as_field_group }}
  <input type="submit" value="Send">
</form>

This reduces the boilerplate code in Django templates and makes the form-rendering markup more reusable. At the same time, it’ll be possible to tweak the default field group template to customize it if necessary.

These are the most significant features coming in Django 5.0. The release will also include several minor improvements, backward-incompatible changes, deprecations, and removals. To find out more details, read the official release notes on the framework’s website.

Are you excited to try out Django 5.0 in your projects?