A couple of months ago, I merged a pull request to improve Strawberry’s error messages and make them more user-friendly. In this article I’ll show the steps I took to improve the user experience of the Strawberry error messages and, hopefully, inspire you to do the same on your projects!

Errors can be quite From u/Naco88 in r/mildlyinfuriating frustrating , especially if they are difficult to understand or lack helpful information.

This is why I spent quite a bit of time improving the error messages in Strawberry, and I’m really happy with the results:

👆 In this screenshot you can see the new Strawberry error messages, they point straight at the error source and provide helpful information to fix the error.

Strawberry is a Python library to create GraphQL APIs. It makes writing a schema for GraphQL pretty straightforward. For example, here’s a small API that exposes a song with its title and artist:

import strawberry

@strawberry.type
class Song:
    title: str
    artist: str

@strawberry.type
class Query:
    @strawberry.field
    def song(self) -> Song:
        return Song(
            title="Strawberry Fields Forever",
            artist="The Beatles"
        )

schema = strawberry.Schema(Query)

Strawberry allows developers to use Python’s type hints in their GraphQL code, making it easier and more intuitive to create GraphQL APIs from Python code. One of the key goals of Strawberry is to have a great user experience, and that’s why I spent some time improving the error messages.

We’ve already seen an example of the new error messages, but let’s see what we had before, and how we improved them. For this, we’ll use the same code snippet we used before, but we’ll add an error to it:

import strawberry

@strawberry.type
class Song:
    title: str
    artist: str

@strawberry.type
class Query:
    @strawberry.field
    def song(self):  # 👀 note the missing return type here
        return Song(
            title="Strawberry Fields Forever",
            artist="The Beatles"
        )

schema = strawberry.Schema(Query)

At first glance it might not be clear where the error is—especially if you are unfamiliar with Python type hints, Strawberry or GraphQL. And our original error message doesn’t help much either:

Traceback (most recent call last):
  File "/tmp/👋/demo.py", line 11, in <module>
    class Query:
  File "/tmp/👋/strawberry/object_type.py", line 253, in type
    return wrap(cls)
  File "/tmp/👋/strawberry/object_type.py", line 239, in wrap
    wrapped = _wrap_dataclass(cls)
  File "/tmp/👋/strawberry/object_type.py", line 102, in _wrap_dataclass
    _check_field_annotations(cls)
  File "/tmp/👋/strawberry/object_type.py", line 80, in _check_field_annotations
    raise MissingReturnAnnotationError(field_name)
strawberry.exceptions.MissingReturnAnnotationError:
    Return annotation missing for field "song",
    did you forget to add it?

We already tried to make this error helpful, and in fact it does tell us that we are missing a return annotation, but it doesn’t tell us where the error is or how to fix it. The traceback lists the file where the error is raised, but that’s not where the original error is, it’s just where the error is handled.

Thankfully, the new Strawberry errors will point straight at the error source, and provide helpful information to fix the error:

  error: Missing annotation for field `song`

       @ demo.py:13

    12 |     @strawberry.field
  ❱ 13 |     def song(self):
                 ^^^^ resolver missing annotation
    14 |         return Song(
    15 |             title="Strawberry Fields Forever",
    16 |             artist="The Beatles"
    17 |        )

  To fix this error you can add an annotation,
    like so `def song(...) -> str:`

  Read more about this error on
    https://errors.strawberry.rocks/missing-return-annotation

This is a huge improvement, and I’m really happy with the results. I also took extra care to make sure that how we implemented the new errors is easy to understand, to maintain and also doesn’t add too much overhead to the library (we’ll see more about this later).

Implementing better errors

There are a few steps involved in showing errors like the one above, but let’s start from what allows us to change how an exception is printed.

If you raise an exception in Python, it’s likely that the output you’ll receive will look something like this:

>>> raise ValueError("Hey there! this is an error! 💥")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Hey there! this is an error! 💥

Which is fine for basic errors, but it becomes difficult to follow when the tracebacks are long (or there are multiple exceptions), especially for beginners.

Oh, by the way, if you love improved errors messages, checkout Python 3.11, they made tracebacks so much better!

Ok, let’s see how we can create better-looking exceptions, and then we’ll see how to print them when they are raised.

”Rich” exceptions

Rich is the best (python) tool for crafting beautiful command line interfaces, so why not use it for exception handling too? Here’s an example:

import rich
import rich.console
import rich.panel

class AnException(Exception):
    def __rich__(self) -> rich.console.RenderableType:
        return rich.panel.Panel("An Exception", title="An Exception")

rich.print(AnException())

Running this code will get us something like this:

╭───────────── An Exception ─────────────╮
│ An Exception                           │
╰────────────────────────────────────────╯

It’s not exactly what I’ve shown you above, but it gives us an idea of how to customize an exception. Unfortunately though, defining __rich__ on an exception won’t do anything when it is raised. This is why we were temporarily using rich.print in our example.

Overriding the default exception handler

Now that we have an exception that can be printed using rich, we need to find a way to do that automatically when an exception is raised. We can do this by overriding sys.excepthook with a function, which will be called every time an exception is raised. Let’s update our example!

import sys
import types
from typing import Optional, Type

import rich
import rich.console
import rich.panel

class AnException(Exception):
    def __rich__(self) -> rich.console.RenderableType:
        return rich.panel.Panel("An Exception", title="An Exception")

def custom_excepthook(
    type: Type[BaseException],
    value: BaseException,
    traceback: Optional[types.TracebackType],
) -> None:
    rich.print(value)

sys.excepthook = custom_excepthook

raise AnException()

Here, we added a custom_excepthook function and assigned it to sys.excepthook. Now all raised exceptions will go through this function and will be printed using rich.print. That’s it, with just a few lines of code we’ve made our exceptions much prettier!

Before we move on, I want to point out that Rich has a built-in function to make tracebacks look better, but it doesn’t use the __rich__ method defined on the exception, so it won’t help us with creating a fully custom error message.

We can improve this, though! We probably don’t want to take over exceptions that we haven’t defined, we only want to use rich for our exceptions. To do this, we can create a unique base class for all our exceptions and restrict the use of rich.print to those cases only. All other scenarios should be managed using the default except hook.

import sys
import types
from typing import Optional, Type

import rich
import rich.console
import rich.panel

class RichException(Exception):
    ...

class AnException(RichException):
    def __rich__(self) -> rich.console.RenderableType:
        return rich.panel.Panel("An Exception", title="An Exception")

def custom_excepthook(
    type: Type[BaseException],
    value: BaseException,
    traceback: Optional[types.TracebackType],
) -> None:
    if isinstance(value, RichException):
        rich.print(value)
    else:
        sys.__excepthook__(type, value, traceback)

sys.excepthook = custom_excepthook

raise AnException()

That’s much better! Now we only use rich for our exceptions, and we don’t override the default behavior for other exceptions.

On making this even better!

We should consider a few extra checks for our custom_excepthook. For example, we can allow users to disable the custom printing with an environment variable. Or make sure that any potential errors in the printing logic doesn’t lead to any more issues.

One important check we added in Strawberry is to make sure that if rich is not installed, we don’t override the default exception hook. We don’t want to force our users to install rich if they don’t want to. One could also decide to only use rich in development, and use the default exception hook in production.

You can see the full implementation on GitHub

Ok, so far, we’ve seen how to create a custom exception and how to print it using rich and a custom sys.excepthook, but usually that’s not enough, we also want our user to be able to see the source code of the error, if there’s one.

Adding the source code of the error

By default, tracebacks already come with the source of an error and a traceback. We have customised our exception handling to print our exceptions using rich, but doing that has hidden the source code of the error. We could probably show the traceback, but that’s not great in term of UX and in Strawberry’s case the traceback and the source of the error aren’t the same in most cases. Let’s get back to our example:

import strawberry

@strawberry.type
class Song:
    title: str
    artist: str

@strawberry.type
class Query:
    @strawberry.field
    def song(self):  # 👀 note the missing return type here
        return Song(
            title="Strawberry Fields Forever",
            artist="The Beatles"
        )

schema = strawberry.Schema(Query)

The exception is raised inside @strawberry.type, as we can kind of see from the original traceback:

Traceback (most recent call last):
  File "/tmp/👋/demo.py", line 11, in <module>
    class Query:
  File "/tmp/👋/strawberry/object_type.py", line 253, in type  # this is where the exception is raised
    return wrap(cls)
  File "/tmp/👋/strawberry/object_type.py", line 239, in wrap
    wrapped = _wrap_dataclass(cls)
  File "/tmp/👋/strawberry/object_type.py", line 102, in _wrap_dataclass
    _check_field_annotations(cls)
  File "/tmp/👋/strawberry/object_type.py", line 80, in _check_field_annotations
    raise MissingReturnAnnotationError(field_name)
strawberry.exceptions.MissingReturnAnnotationError:
  Return annotation missing for field "song",
  did you forget to add it?

So, although the error is raised in @strawberry.type, the source of the error is in the song field, since that’s where the return type is missing.

Fortunately, since we already override how the exception is show we can also customise the logic that finds the source of the error. In this case we can use the field_name and the class object (cls) to find the source of the error!

Finding the file in which a class was defined

Now that we have cls and field_name we can use some Python magic to find where the class was defined. To do so we can use the __module__ attribute on the class, this attribute contains the name of the module where the class was defined, once we have that we can try using sys.modules to get the module object, if we find it then we are pretty much done, since the module object has the __file__ attribute that contains the path to the file where the module was defined.

If we don’t have a module we can fallback to using the importlib.util.find_spec function to try and find this module. Ideally this case won’t happen often, but there were some cases in which I had to use it.

Once we have the module name, we need to find the line where the object was defined.

Finding the line where a class was defined

This is the trickiest part, we need to find the line where the class was defined and then find the line where the field was defined. In Python there’s a built-in function to do this, inspect.getsourcelines which returns the source code of the object and the line number where the object was defined.

Unfortunately, I wasn’t able to get this function working consistently for all use cases, so I wrote some code to use the AST of the module to find the class (or any other object) that I’m looking for.

Using the AST to find the source of an error

The AST is a great tool to find the source of an error, it’s a tree that contains all the information about the code, including the line numbers. We could use the ast.parse function to parse the source code of the module and then use the ast.walk function to traverse the tree and find the class that we are looking for.

But doing that would take a fair amount of code and I also found the AST module to behave differently in different Python versions, so I decided to use libcst, a library that parses Python code into a CST (Concrete Syntax Tree) which is similar to the AST (in our use case it doesn’t matter if it’s an AST or a CST).

With libcst you can create matchers that can be used to find the class or object you’re looking for. For example, if we want to find a class we can use the ClassDef matcher, and if we want to find a function we can use the FunctionDef matcher.

I’m going to avoid showing you the full code here, since there’s some boilerplate and it’s not the most interesting part of this post, but you can see the full implementation on GitHub

I’d love to extract this part of Strawberry’s code base into a separate library that can be used by other projects, but I haven’t had the time to do it yet, if you’re interested in something like this, please considering ✨ sponsoring me ✨ , it would help me a lot in finding time to work on this!

Showing the source of the error

Now that we have the source of the error, we can show it to the user! As I said before, we create a base exception class that all Strawberry’s exceptions inherit from, this class implements the __rich__ method, similar to how we did in the previous sections.

It also delegates to subclasses the responsibility of finding the source of the error. You can find the full implementation on GitHub

We also implemented some custom logic to add the error message on right line of the source code. We are using Rich’s Syntax class to show the source code, which doesn’t support add arbitrary text on a specific line, so we had to implement this logic ourselves.

Conclusion

I truly care about the developer experience of Strawberry and I think that having good error messages is a key part of it, I’m sure in future we’ll find more ways to improve the error messages or the developer experience in general and I can’t wait to share them with you!

I hope you enjoyed this post, I know I’ve skipped some details, so if you’re interested in knowing more please send me a message!

If you want to learn more about Strawberry, you can check out it here or join our Discord server!