Classes and Objects

You can't discuss classes without speaking about objects, and you can't talk about objects without mentioning classes. So, off we go.

States and Behaviors

As we mentioned earlier, we use classes to create objects. We typically focus on state and behavior when defining a class. State refers to the data associated with an individual class instance; behavior is what class instance objects can do, i.e., what methods an object can call.

For example, given our GoodDog class from the last chapter, we can create two GoodDog objects: one for Sparky and one for Rover:

sparky = GoodDog('Sparky', 5)
rover = GoodDog('Rover', 3)

sparky and rover are GoodDog objects, but they contain different information, such as their names and ages. We use instance variables to track this state information.

While these objects have different states, they have identical behaviors. For example, both GoodDog objects should be able to bark, run, fetch, and perform other common behaviors of good dogs. We define these behaviors as instance methods in the class definition. Instance methods provided by a class are available to all class instances.

In summary, instance variables keep track of the state, while instance methods expose behavior for objects. We'll spend the rest of this chapter learning how to work with objects, instance variables, and instance methods.

Object Scope

Object scope refers to the methods and instance variables an object can access. This is akin to discussing global and local scope instead of identifier scope. With identifier scope, you want to know which parts of a program can see a particular identifier. With global and local scope, you want to know what identifiers can be seen at any given point in a program. With object scope, we want to know what methods and states the object can see.

Object scope has two main components:

• The methods in the class. This includes any methods acquired by the class via inheritance or mix-ins. (We'll discuss inheritance and mix-ins later.)
• The instance variables associated with the object. This includes any instance variables acquired via inheritance.

Notice that instance methods belong to their class. Class instances can access those methods, but the methods aren't part of the objects. They are shared by the class.

Instance variables, however, belong to objects. That may seem strange since most methods use and manipulate instance variables. At first glance, they would seem to be part of the class. However, they aren't; they belong to the instance. The methods give values to the instance variables, but those values belong to the object. That's why objects can have states that differ from other instances of the same type.

In short, any object can call any method the class provides; every method can access the object's instance variables. Thus, all instances of the same type can access the same methods. However, an object can only access its own state.

Object Instantiation

Let's start fresh with a new GoodDog class. First, let's create its __init__ method.

class GoodDog:

    def __init__(self):
        print('This object was initialized!')

sparky = GoodDog() # This object was initialized!

The __init__ method gets called every time you create a new object. As you may recall, that's the final step when instantiating an object. The first step is to call the constructor, e.g., GoodDog().

The constructor first calls the static method __new__. (We'll discuss static methods shortly.) This method allocates memory for the object and returns an uninitialized object to the constructor.

The constructor next initializes the object by calling __init__. That is, __init__ initializes the instance variables the object needs in its initial state.

In our GoodDog example, we call GoodDog(), which, in turn, calls GoodDog.__new__. The __new__ method returns the new object, which the constructor subsequently uses to call __init__. Our __init__ method doesn't do anything terribly interesting; it just prints a message so we can see that it ran.

As explained in the previous chapter, __init__ is frequently called the constructor. However, a better name may be the initializer or the instance constructor. We will usually call it the initializer, or more succinctly, __init__.

Instance Variables

Now that we know how to use constructors in Python let's update __init__ to initialize a GoodDog object with some state, like a name and age.

class GoodDog:

    def __init__(self, name, age):
        self.name = name
        self.age = age

sparky = GoodDog('Sparky', 5)

There are several things to notice here:

• The __init__ method has name and age parameters. That lets us give each new GoodDog a name and age.
• Adding parameters to __init__ means you must provide arguments corresponding to those parameters when calling the constructor. On line 7, we pass the string 'Sparky' and the integer 5 as the arguments. They get assigned to name and age respectively.
• self.name and self.age are instance variables. Every GoodDog object will have appropriate values for these variables.

Instance variables keep track of information about the state of an object. In the above code, the self.name instance variable for the sparky object is the string 'Sparky'; it is part of the object's state. Similarly, self.age contains 5; Sparky's age. Suppose we create another GoodDog object with rover = GoodDog('Rover', 3). In this case, rover.name gets set to string 'Rover' while the integer 3 is assigned to rover.age. Every object's state is distinct (though not necessarily unique). Instance variables are how we keep track of the state.

Instance variables don't go away when __init__ finishes running. They continue to exist as long as the object exists unless explicitly deleted.

Instance Methods

Right now, our GoodDog class doesn't do anything interesting. Let's give it some behavior.

class GoodDog:

    def __init__(self, name, age):
        self.name = name
        self.age = age

    #highlight
    def speak(self):
        return 'Arf!'
    #endhighlight

sparky = GoodDog('Sparky', 5)
#highlight
print(sparky.speak())
#endhighlight

Notice that the speak method definition needs one parameter, conventionally named self. All instance methods must have a self parameter. You can use a name other than self if you don't mind shunning by other Python programmers.

When you run this program, it prints Arf!. We told Sparky to speak, and he did.

Suppose we have another GoodDog object:

# Omitted code

#highlight
rover = GoodDog('Rover', 3)
print(rover.speak())            # Arf!
#endhighlight

Our second object can perform the same GoodDog behaviors as the first. Again, all instances of a class have the same behaviors, though they may contain different states. Here, the differing state is the name and age, which don't play any part in the speak method.

Suppose we want to not just say Arf! but Sparky says arf! instead? In our instance methods, we have access to the instance variables associated with an object. Thus, we can use string interpolation and change our speak method as shown below:

class GoodDog:

    def __init__(self, name, age):
        self.name = name
        self.age = age

    def speak(self):
        #highlight
        return f'{self.name} says arf!'
        #endhighlight

sparky = GoodDog('Sparky', 5)
print(sparky.speak())  # Sparky says arf!

rover = GoodDog('Rover', 3)
print(rover.speak())   # Rover says arf!

The behavior of speak is now dependent on the state of the GoodDog instance object used to invoke it.

Privacy

Suppose we want to print sparky's name. We could try the code below:

# Omitted code

print(sparky.name)            # Sparky

Whoa! It works! You can even reassign the instance variable:

# Omitted code

sparky.name = 'Fido'
print(sparky.name)            # Fido

If the value referenced by the instance variable is mutable, you can also mutate it.

This sends shivers up the spines of many experienced OOP developers. You can view or change any object's instance variables by just referencing the variable name. You can even delete or change methods. This can lead to all kinds of problems. Here are a few of those problems:

• Unexpected instance variable values can lead to incorrect or unexpected behavior of the class instances.
• The class developer may change the implementation in the future, which may break your code.
• Incorrect or modified attributes can lead to unanticipated security problems.

What's the poor class developer to do? The Python community shrugs it off as, "We are all responsible users." (You may hear that phrased another way; we won't use that alternative phrasing.) If a class user wants to access or modify an attribute directly, they can, even if those attributes aren't described as part of the class's interface. As the class's developer, you're not obligated to support undocumented or improper use of attributes. However, that doesn't mean you won't waste time debugging such issues. Unfortunately, you'll get little satisfaction in subsequently telling the user that they are misusing the class.

That said, there are some things you can do to discourage users using your "private" attributes. The simplest approach is to rely on the convention of marking instance variables and methods for internal use by naming them with a single leading underscore:

class GoodDog:

    def __init__(self, name, age):
        #highlight
        self._name = name
        self._age = age
        #endhighlight

    def speak(self):
        #highlight
        return f'{self._name} says arf!'
        #endhighlight

    def _dog_years(self):
        return self._age * 7

    def show_age(self):
        print(f'My age in dog years is {self._dog_years()}')

# Omitted code

This doesn't prevent messing around with these internal use attributes, but it's a clear signal to the user that they're playing with fire. The single underscore convention tells the user they're messing with something they shouldn't. If they go ahead and do so anyway, it's at their own risk. Fortunately, most Python programmers will happily observe this convention.

You can also use two leading underscores (but no trailing underscores), which causes name mangling:

class GoodDog:

    def __init__(self, name, age):
        #highlight
        self.__name = name
        #endhighlight
        self._age = age

    def speak(self):
        #highlight
        return f'{self.__name} says arf!'
        #endhighlight

sparky = GoodDog('Sparky', 5)

sparky.__name = 'Fido'
print(sparky.__name)         # Fido
print(sparky.speak())        # Sparky says arf!

sparky._GoodDog__name = 'Fido'
print(sparky._GoodDog__name) # Fido
print(sparky.speak())        # Fido says arf!

On lines 12-14, we attempted to change the name using the unmangled name, __name. Initially, this seemed to work. However, sparky.speak() didn't reflect the name change. Instead, Python created a new __name instance variable that speak doesn't know about.

Lines 16-18 show how easy it is to defeat name mangling. Mangled names are formed by prepending an underscore and the class name to the unmangled name.

There's little benefit to preferring the single or double underscore convention. Thus, we recommend choosing one style and sticking with it. Our convention will be to use single underscores unless we wish to talk specifically about the double underscore convention.

Getters and Setters

Since we can't prevent unrestricted access to instance variables, the next best approach is to provide getter and setter methods for the instance variables a user might want to access or modify. Getters and setters are common in OOP; they are methods that provide controlled access to an object's attributes. Getters retrieve attribute values, while setters assign attributes to new values.

Let's take a look at an example:

class GoodDog:

    def __init__(self, name, age):
        self._name = name
        self._age = age

    def speak(self):
        return f'{self._name} says arf!'

    #highlight
    def name(self):
        return self._name

    def set_name(self, new_name):
        if not isinstance(new_name, str):
            raise TypeError('Name must be a string')
        self._name = new_name

    def age(self):
        return self._age

    def set_age(self, new_age):
        if not isinstance(new_age, int):
            raise TypeError('Age must be an integer')
        if new_age < 0:
            raise ValueError("Age can't be negative")
        self._age = new_age
    #endhighlight

sparky = GoodDog('Sparky', 5)
print(sparky.name())          # Sparky
print(sparky.age())           # 5
sparky.set_name('Fireplug')
print(sparky.name())          # Fireplug
sparky.set_age(6)
print(sparky.age())           # 6

sparky.set_name(42)
# TypeError: Name must be a string

sparky.set_age(-1)
# ValueError: Age can't be negative

In this code, we've used the underscore conventions to mark self._name and self._age for internal use. However, we've created getter methods name and age to retrieve the self._name and self._age values, respectively. We've also defined setter methods named set_name and set_age to change a GoodDog's name and age. To avoid unexpected values being submitted, we raise an exception if the name isn't a string or the age isn't a non-negative integer.

Users of your class can still choose to ignore the underscore convention. However, they no longer need it for any legitimate purpose. They can use the getters and setters.

Getters conventionally have the same name as the associated instance variable without leading underscores. Setters conventionally prefix the same name with set_.

Properties

A more Pythonic way to create getters and setters is to use the @property decorator: Decorators are a fairly advanced concept in Python. They are, in fact, methods that modify other methods (the name and age methods seen above). They have many uses in Python; we'll see a few more later, but we won't learn to create our own in this book. We'll just show you how to use them.

The @property decorator is used to create getter methods for an instance variable. When you apply @property to a method named foo, @property creates a secondary decorator named @foo.setter; this secondary decorator is used to create setter methods. (Thus, you can have a getter without a setter, but you can't have a setter without a getter.)

Let's see how it's done:

class GoodDog:

    def __init__(self, name, age):
        self.name = name
        self.age = age

    def speak(self):
        return f'{self.name} says arf!'

    #highlight
    @property
    #endhighlight
    def name(self):
        return self._name

    #highlight
    @name.setter
    def name(self, name):
    #endhighlight
        if not isinstance(name, str):
            raise TypeError('Name must be a string')

        self._name = name

    #highlight
    @property
    def age(self):
    #endhighlight
        return self._age

    #highlight
    @age.setter
    def age(self, age):
    #endhighlight
        if not isinstance(age, int):
            raise TypeError('Age must be an integer')

        if age < 0:
            raise ValueError("Age can't be negative")

        self._age = age

sparky = GoodDog('Sparky', 5)
#highlight
print(sparky.name)          # Sparky
print(sparky.age)           # 5
sparky.name = 'Fireplug'

print(sparky.name)          # Fireplug
sparky.age = 6

print(sparky.age)           # 6

sparky.name = 42  # TypeError: Name must be a string

sparky.age = -1   # ValueError: Age can't be negative
#endhighlight

With this code, we seem to have two different methods called name and two more named age. The decorators, @property, @name.setter, and @age.setter, make them distinct. The @property prior to the first name method creates the @name.setter decorator, while the one prior to the first age method creates the @age.setter decorator.

Using these decorators means we no longer need () when accessing the getter and setter. We can also use standard assignment syntax to give an instance variable a new value.

Getters created with the @property decorator are known as properties. A setter is simply a property whose value can be reassigned.

There are a couple of conventions you can see in the above code:

• Only the @property, @name.setter, and @age.setter methods use the underscored name. All other references that look like instance variable accesses are, in fact, calling the property methods.
• The name of the setter argument is, conventionally, the same name as used by the method. There is no need to use new_name or new_age.

When to Use Properties

Despite the "We are all responsible users" sentiment in the Python community, properties are widely considered good practice, provided they aren't overused. As a rule, you should use properties when:

• you want to strongly discourage misuse of the instance variables.
• you want to validate data when your instance variables receive new values.
• you have dynamically computed attributes.
• you need to refactor your code in a manner incompatible with the existing interface.
• you want to improve your code readability, and properties can help.

If you don't need properties to satisfy a specific problem, you shouldn't use them.

When you use properties, use the single or double underscore convention for the associated instance variables.

In the remainder of this book, we will mostly avoid properties, mostly to reduce the amount of code we need to show. However, please don't assume you don't need to understand properties.

Class Methods

Thus far, all the methods we've created are instance methods. They are methods that pertain to a class instance. There are also class-level methods called class methods.

Class methods provide general services for the class as a whole rather than the individual objects. We usually use the class to invoke the method. However, Python also lets you invoke class methods with instance objects. That's a little confusing, though, so you should use the class if possible.

We use the @classmethod decorator to create a class method. Class methods require at least one parameter: the class itself. By convention, the first parameter is named cls:

class GoodCat():

    @classmethod
    def what_am_i(cls):
        print("I'm a GoodCat class!")

When we call the class method, we use the class name GoodCat followed by a period and the method name:

GoodCat.what_am_i()    # I'm a GoodCat class!

Why do we need a class method for this? This admittedly contrived example doesn't need a class method. However, it demonstrates how to create and use them. Also, class methods are where we usually put functionality that doesn't deal with class instances. Since our method has no reason to use the instance variables, we use a class method instead.

There are a variety of ways to call class methods. We can't provide explicit advice for what works best in every situation, but here are some guidelines you can use:

• If you need to call a class method from within another class method of the same class, you can use the cls argument as the caller for the second method:

  class Foo:
  
      @classmethod
      def bar(cls):
          print('this is bar')
  
      @classmethod
      def qux(cls):
          print('this is qux')
          cls.bar()
  
  Foo.qux()
  # this is qux
  # this is bar
  

• When you want to call a specific class method from outside the class that contains the class method, use the class's name to call it, as we did with Foo.qux() above. If you call a class method without using the explicit class name, Python will use the inferred class and the method resolution order (MRO) to determine which class method it should use. (We'll discuss the MRO later.)

• If you have an instance object, obj, of a class that has a class method, you can invoke that method by using type(obj), obj.__class__, or even obj as the caller. You can also use self inside a method, as we show below:

  class Foo1:
  
      @classmethod
      def bar(cls):
          print('this is bar in Foo1')
  
      def qux(self):
          type(self).bar()
          self.__class__.bar()
          self.bar()
          Foo1.bar()
  
  class Foo2(Foo1):
  
      @classmethod
      def bar(cls):
          print('this is bar in Foo2')
  
  foo1 = Foo1()
  foo1.qux()
  # this is bar in Foo1
  # this is bar in Foo1
  # this is bar in Foo1
  # this is bar in Foo1
  
  foo2 = Foo2()
  foo2.qux()
  # this is bar in Foo2
  # this is bar in Foo2
  # this is bar in Foo2
  # this is bar in Foo1
  

The main difference between using type(self).bar() vs. self.__class__.bar() is readability. We believe self.__class__.bar() is more readable despite needing 3 additional characters, but that's strictly a personal opinion.

We strongly discourage using the obj.bar() syntax for class methods. You lose any indication that you're calling a class method.

Class Variables

Instance variables capture information related to specific class instances. Similarly, class variables capture information about the class. We initialize class variables in the main class body, usually at the top of the class. We can access and manipulate them with both instance and class methods.

class GoodCat:

    counter = 0                  # class variable

    def __init__(self):
        GoodCat.counter += 1

    @classmethod
    def number_of_cats(cls):
        return GoodCat.counter

class ReallyGoodCat(GoodCat):
    pass

cat1 = GoodCat()
cat2 = GoodCat()
cat3 = ReallyGoodCat()

print(GoodCat.number_of_cats())        # 3
print(GoodCat.counter)                 # 3
print(ReallyGoodCat.number_of_cats())  # 3
print(ReallyGoodCat.counter)           # 3

In GoodCat, we have a class variable name, counter, which we initialize to 0. We want to use this variable to keep track of how many cats we have, so on line 6, we increment it by 1. Since Python calls __init__ every time we instantiate a new object, it's a great place to increment counter.

Note that we opted to use the explicit class name in __init__ instead of using self.__class__.counter or cls.counter. If we use self.__class or cls, we end up with some unusual results:

class GoodCat:

    counter = 0                  # class variable

    def __init__(self):
        #highlight
        self.__class__.counter += 1
        #endhighlight

    @classmethod
    def number_of_cats(cls):
        #highlight
        return cls.counter
        #endhighlight

class ReallyGoodCat(GoodCat):
    pass

cat1 = GoodCat()
cat2 = GoodCat()
cat3 = ReallyGoodCat()

#highlight
print(GoodCat.number_of_cats())        # 2
print(GoodCat.counter)                 # 2
#endhighlight
print(ReallyGoodCat.number_of_cats())  # 3
print(ReallyGoodCat.counter)           # 3

Here, the counter tells us that we only have 2 GoodCat objects but 3 ReallyGoodCat objects. We only see 2 GoodCat objects since cat3 ended up creating a counter in ReallyGoodCat instead of incrementing the counter in GoodCat. We also see 3 ReallyGoodCat objects because of inheritance.

If we want to count all GoodCat objects, including any instances of ReallyGoodCat or other subclasses, we need to increment GoodCat.counter explicitly instead of self.__class__.counter. Otherwise, we'll end up incrementing a counter variable in the subclass. For the same reason, we also refer to GoodCat.counter in the number_of_cats method.

As with class methods, you can use cls.variable, type(obj).variable, obj.__class__.variable, and obj.variable to access class variables. Once again, you should use the explicit class name syntax if you want to make sure you're accessing a class variable in a specific class.

Class Constants

Some classes have variables you never want to change once the class is defined. For this, you can use class constants. Class constants have the same naming conventions as ordinary constants. Like those constants, they are only constant by convention. Python doesn't enforce constancy:

class GoodCat:
    CAT_YEARS = 5

    def __init__(self, name, age):
        self.name = name
        self.age = age

    def human_age(self):
        return self.age * GoodCat.CAT_YEARS

cocoa = GoodCat('Cocoa', 4)
print(cocoa.human_age())

Here, we used the constant CAT_YEARS to calculate the equivalent human age for a cat. CAT_YEARS is a variable that never changes, so we use the SCREAMING_SNAKE_CASE convention to show that it is meant to be constant.

Okay, we know cats don't age at the same rate throughout their lives. The 5 years is merely an approximation.

As with class methods, you can use cls.CONSTANT, type(obj).CONSTANT, obj.__class__.CONSTANT, and obj.CONSTANT to access class constants. Once again, you should use the explicit class name syntax if you want to make sure you're accessing a class constant in a specific class.

More About self

We've mentioned and used self fairly often so far. Let's dive a little deeper so you can understand what exactly self is and what it represents.

The most important thing to understand about self is that it always represents an object. What object, though? It's the calling object for a method. For instance, if we call cocoa.human_age, the object represented by cocoa is a GoodCat object we created by calling the GoodCat constructor. Thus, when we access self.age in the GoodCat.human_age method, self.age refers to the value assigned to the cocoa.age instance variable.

Things become slightly more confusing when dealing with inheritance. Consider the following code based on some code we saw in the Object Model chapter earlier in this book:

class Pet:

    def __init__(self, name):
        self.name = name

    def speak(self, sound):
        print(f'{self.name} says {sound}!')

class Cat(Pet):

    def speak(self):
        super().speak('meow')

cheddar = Cat('Cheddar')
cheddar.speak()

Looking at this code, you might conclude that self in self.name refers to an object of the Pet class. That's not entirely wrong. However, the actual calling object is a Cat object. It's that object that self refers to on line 7. It happens that a Cat object is also a Pet object, which explains our first sentence.

This also applies to line 4, though we never directly called the __init__ method from the Cat class. However, Python uses a Cat object to call __init__.

By the way, that invocation of super() on line 12 returns an object that lets you call methods from the superclass of an object. Thus super().speak('meow') calls the speak method from the Pet class. We'll see more about super() later.

It's worth repeating that the name self is a convention. The first parameter defined for any instance method always represents the calling object, no matter what name you use. Regardless, always use self when writing Python instance methods. Your teammates and successors won't yell at you.

More About cls

The first parameter of a class method, conventionally named cls, always represents a class. Usually, that's the class used to invoke the method. For instance, if we call the GoodCat.number_of_cats method from earlier, we're using the GoodCat class to call the method. Thus, when we access cls.counter in the method, it refers to the number of GoodCat objects created.

cls is nearly identical to self in almost all respects. However, it conventionally references a class rather than an ordinary object. In Python, though, classes are instance objects, too! They are instantiated from the type class. Theoretically, there is no difference between cls and self. Nevertheless, use cls when defining a class method and self for instance methods.

The fact that classes are objects suggests something: you can use super() to reference a class method in a superclass. This isn't as useful as you might think, but it's worth remembering. For now, here's a contrived example:

class Animal:

    @classmethod
    def make_sound(cls):
        print(f'{cls.__name__}: A generic sound')

class Dog(Animal):

    @classmethod
    def make_sound(cls):
        super().make_sound()
        print(f'{cls.__name__}: Bark')

Dog.make_sound()
# Dog: A generic sound
# Dog: Bark

Note that cls.__name__ is evaluated as 'Dog' in both methods. This shouldn't surprise us: we used the Dog class to invoke the method.

It's worth repeating that, as with self, the name cls is a convention. The first parameter defined for any class method always represents the calling class, no matter what you call it. Regardless, always use cls when writing Python class methods. Your teammates and successors will appreciate your acceptance of conventional practices.

Static Methods

You'll often encounter methods that belong to a class, but don't need access to any class or instance attributes. As a result, they don't make sense as either class or instance methods. Instead, they usually provide utility services to the instance or class methods, or to the users of the class. These methods are called static methods.

To define a static method, you use the @staticmethod decorator followed by a function definition that doesn't use a self or cls parameter. For instance, suppose you're developing a game that displays the game rules to the player. The rules don't need to know anything about the game instance or the class, but you might still want them to be part of the class. This is a good spot for a static method:

class TheGame:
    # Game playing code goes here

    def play(self):
        pass

    @staticmethod
    def show_rules():
        print('These are the rules of the game')
        # The rules go here.

TheGame.show_rules()

game = TheGame()
game.play()

Our static method, show_rules, begins with the @staticmethod decorator and lacks a self or cls parameter. We can invoke it using the class method invocation syntax (TheGame.show_rules() in this case) or any other syntax that works for class methods. The only real difference between a class method and a static method is that the static method doesn't have a cls argument it can use.

Not all methods that can be made into static methods should be. For instance, a static method can't be easily converted to an instance method without requiring code changes elsewhere. If there's a reasonable chance that a static method may one day require access to instance or class state, then the method may not be suitable for use as a static method.

Static methods are often meant for internal use only, i.e., helper methods for your class's instance and class methods. They are also suitable for clarifying intent: use a static method when you want to be clear that the method doesn't use or modify the object or class state.

Summary

This chapter covered a lot of information about classes, objects, methods, self, and cls. You are becoming much more proficient in the language. You are growing as a Python programmer. This is where things finally start to get fun.

In this chapter, we covered:

• states and behaviors
• object scope
• object instantiation
• instance variables and methods
• getters and setters
• class methods, variables, and constants
• static methods
• self and cls

If you've never seen OOP before, this chapter should've been a doozy. We need lots of practice to truly understand it. Please don't skip the exercises; OOP is a vital piece of foundational knowledge that you'll need to truly understand Python code and its frameworks.