Understanding namespaces in Python 3

Introduction

One day at work you develop an important module that is going to be used in the next weeks. In your first task using your module you notice something, tests are no passing and you need to fix them before working in other tasks. As your module is already working you check your implementation, from parameters to names, everything seems OK, then a colleague comes and tells you "You have a conflict of names between you module and another module", because yet no one is using you module you go an change the conflicting names, the logical errors are fixed and you can go to the next task.

What you have witnessed is the result of namespace ruling, it is important to know this concept because it revolves around having an scope between all the names present in a project, so in this article I am discussing about it.

Prerequisites

To successfully follow this article it is a must to have these requisites in the machine you are working with:

• Python environment installed.
• Text editor of preference.

Names in Python

Everything in Python is an object and these objects are stored in RAM at run time, every object posses an id that we can use to retrieve our object, but in practice using the id to work its not recommended (and you should not do it) because we do not know what we are using until it is called, to resolve this we need mechanism to identify our object, this is the fundamental of why names exists in programming languages in general.

So a name in Python is the identifier used to refer to an object, in other words a name is what we colloquially call name of the variable or just variable.

To understand how names work lets do an experiment, create a file called name.py in your folder of choice and paste the following code:

x = 9
print("Id of 9: ", id(9))
print("Id of x: ", id(x))

y = 9
z = x
print("Id of y: ", id(y))
print("Id of z: ", id(z))

The code is about reporting the id of some objects at different points in execution runtime. Executing the script leads to the following output:

Id of 9:  140611653450368
Id of x:  140611653450368
Id of y:  140611653450368
Id of z:  140611653450368

Note: The id will vary each time you execute the file, so it is highly possible that your output will not be the same as this post.

Note that the reported id is the same in all variables, this phenomenon implies the following information:

• The id of the object in raw state is the same as the id of the variable that points to it. Which is the case for first and second result.
• After executing more instructions, passing 9 to other variable lets us see that it have the same id that we saw before. This is the case for third result.
• And the id is kept even after passing it to another variable, as seen in the last report.

With this information it is concluded that we are dealing with the same object all the time, and we are passing its id or reference as known in languages like C. So a variable or name is the identifier that points to an object instead of storing it. This conclusion will help later to ground the concept of namespaces.

Defining namespaces

Now that we know what a name is, namespaces can be defined. At first one can think that as the term suggests, the namespaces is a collection of names, and it is correct, but Python expands this definition to "the mapping of every name with its correspondent object". So in other words, the namespace is the mechanism that points a name and only one object, whether is an object, a class or a function in a manner that we can use that object with invoking its correspondent name.

There are three types of namespaces:

• Built-in namespace: The starting namespace of the program, some examples of names hold are: print, input, str, int, float, list and dir.
• Global namespace of the module: This namespace lives only in each module or in other words this maps all the names of a Python file.
• Local namespace: This lives inside of a function that again this function lives inside a module.

Back in the definition a concept was mentioned, "mapping", that means that the namespace is actually a dictionary object, unfortunately, this dictionary is abstracted and cannot be accessed in a formal way, there is another tool to list all the names of an object, this tool is the dir function built-in in Python, with this one can see directly the namespace of the project in different moments.

To understand some characteristics of namespaces we will execute some code that will use dir function at different moments, so in the same directory we have been working, we will create two file, the first file has this content:

import external

print('Global namespace ', dir())

def function1(foo, bar):
    return "function1"

def _function2(foo, bar):
    return "function2"

def __function3(foo, bar):
    return "function3"

external.sumn(5, 9)

print('Global namespace updated ', dir())

For the second file we are having this content:

import math
print('Module namespace ', dir())

def sumn(a, b):
  print('Local namespace ', dir())
  return a + b

After we have pointed the content of both files we are executing them to show how namespaces work, this time you should have the exact output as follows:

Module namespace  ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'math']
Global namespace  ['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'external']
Local namespace  ['a', 'b']
Global namespace updated  ['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__function3', '__loader__', '__name__', '__package__', '__spec__', '_function2', 'external', 'function1']

Before directly analyzing namespaces there is something that should catch your attention. The first namespace to be reported is from the module, and this is logical, after all the next step from importing should be reading that module so we are able to use its content later.

Now lets review the contents of the output:

• The first to appear is the namespace of the module, we can see some internal attributes and at the end of the list we got the attribute math, if we go back we can remember that math module was imported, but it is strange that the function sumn is not appearing in the list.
• The second one is the global module and it is almost identical to the module one, here we got two more attributes __annotations__ and external which is the name of our module.
• In the third namespace we have something interesting, we are only getting two attributes, as we said earlier, local namespaces only record the names of an specific function, and if you go back to the file you will note it we only have two attributes.
• And last but not least we are reporting again the global namespace and there is extra information now, we see the names of the functions and the module imported in the main file.

Now putting together this information we get these conclusions of the workings of namespaces:

• Namespaces are isolated: We clearly see it in all reports, the first one tell us that we are importing math, but it is not telling us the actual content of math, the second one is telling us that external is imported, but we are not able to see the sumn name, the third one cannot see the math import even that a function exists in the module, and at least the global namespace again is telling us the names we have, but not the content of them.
• Namespaces are created and modified when needed: Lets compare the second and fourth report, each one is reporting a global namespace, actually is the same namespace, but both have different content, the difference remains in the amount of executed content, what we are seeing is that the name is added to the namespace until it is defined.

Now that we fully understand what and how a namespace works lets talk about some little but important topics to have in mind when naming objects.

Conflicts at naming

The dynamic nature of naming in Python lets us change the name of an object in the fly, it means that at different times in execution a given name can point to different objects to, this characteristic can lead to ambiguous objects if not given enough care. To visualize this lets point to two modules in the standard library.

Now we are going to use the Python Shell, so execute the python command in a terminal window.

The Python Shell will enter and show this:

Python 3.6.2 (default, Jul 20 2017, 03:52:27)
[GCC 7.1.1 20170630] on linux
Type "help", "copyright", "credits" or "license" for more information.

We will then import math and also show the content of math with dir:

import math
dir(math)

Then the contents of math are reported:

['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']

Next thing to do is import cmath and again show the contents of cmath with dir:

import cmath
dir(cmath)

This time the following content is reported:

['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atanh', 'cos', 'cosh', 'e', 'exp', 'inf', 'infj', 'isclose', 'isfinite', 'isinf', 'isnan', 'log', 'log10', 'nan', 'nanj', 'phase', 'pi', 'polar', 'rect', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau']

The math and cmath modules refer to math operations, math refers to operations with real numbers and cmath to operations with complex numbers. As we can see there are a lot of "shared" operations in both modules, this can lead to strange behavior if we apply bad practices at importing.

In Python there are three ways of importing a module:

• import module
• from module import attribute
• form module import *

With the first you import the name of the module, to access a function from it you will need to access it by module.attribute notation, but this is not the case to second and third imports, with the second you are importing directly the name of the attribute to your namespace and with the third you are importing all attributes to your namespace.

So if you happen to import two equal names from two different modules, like the ones from math and cmath modules, you are going to be left with one name of the last one you imported, for example if we import acos from math and then we import acos from cmath we are going to have the attribute of cmath, because we are having the same name, Python has to overwrite the pointing from math to cmath, this is the same thing as having a variable and assign it values multiple times with multiple objects.

With this the content of namespaces is almost covered, but there one last catch to know.

Scopes

A scope is a region in which Python can directly access a name, it means that the notation module.attribute is not needed to reach a name. Namespaces are bound to scopes, when resolving a name Python has to look from inwards to outwards passing from local, to the next enclosing area like function nesting, to global and at last to built-in. This exact behavior is also seen when using variables around the program.

Conclusion

With this we reach the end of this article, so now you should understand namespace ruling in Python, with this you do not only get a grasp of how a name of a variable works in Python, but also you understand how to organize names in code in order to avoid ambiguos functions.