Are you a Community Developer? Have you previously written Skills for the Mycroft ecosystem? Know Python, and considering writing Mycroft Skills?
Then you need to read this blog post to understand a key change to Python language support in Skills, and what you need to do to ensure the Skills you maintain are compliant.
Over time, programming languages evolve to meet the needs of their community. That evolution often means that newer versions are not backward compatible. This blog post by core Python developer Brett Cannon explains in more detail why Python moved to version 3. When it began development in late 2015, Mycroft software was still in its infancy – as was Python 3.4+. The decision was made at the time to standardize on Python 2.7. Over the intervening two and a half years, Python 3.4+ has become the de facto standard for Python software development. If Mycroft does not move now to support Python 3.4+, we will become further and further behind the industry standard.
Python 2.7 and Python 3.4+ are not interchangeable; code that works in Python 2.7 might not work in a Python 3.4+ environment. This means we essentially have to choose a time at which to “rip the band-aid off” and drop support for Python 2.7 and adopt Python 3.4+, otherwise we would need to support both versions of the language side by side; this support overhead is not maintainable.
Python 3.4+ introduces several new features that make development easier and less error prone. Key among them is out-of-the-box Unicode support – which is a must-have for Mycroft being able to support languages other than English in the future. Python 3.4+ also handles many functions which return array-like data structures differently, with many now returning iterable objects instead of list data types; this feature provides additional flexibility in data handling. This blog post, from Sebastian Raschka, highlights other key language differences, and is well worth a read. Lisa Tagliaferri from Digital Ocean also has a great write-up on the changes.
From the 18.2.6b release,
mycroft-core is switching to Python 3.4+. If you have Skills that have not been tested under Python 3.4+, then they may not work as intended from version 18.2.6b onwards. We recommend that you take the following action:
- Implement automated testing for your Skill– We’ve recently implemented automated testing tools to help you ensure that your Skill works as intended, and which helps Mycroft assure the quality of Skills that are made available to end users. From 18.2.6b, the automated testing tools will test against Python 3.4+.
- Run your Skill through an automated tool that checks Python 2.7 => Python 3.4+ changes – there are several of these tools around, however we’ve had best results with this one from Juan Carlos Garcia.
If you haven’t already, it’s worth reading up on our Skill Branching changes as well.
In particular, we’ve seen the following common issues:
- Problems handling binary data – because Unicode is default in Python 3.4+, we’ve seen problem in handling binary data because it’s not clear whether a variable is a string or a Unicode object. The
basestringobject was deprecated in Python 3, and you should now use the
strobject. This is a really great read from Python core maintainer, Guido van Rossum, on Text vs Data in Python 3. Sebastian Raschka also has a useful comparison.
- Integer division – In Python 2.7, division of two integers always returned an integer value. For example,
5 / 2 == 2in Python 2.7, however in Python 3.4+, division of two integers returns a float.
5 / 2 == 2.5. Read this howto explainer from Brett Cannon.
- Removed and renamed modules – often in your code you will use the
importstatement to import a Python module for additional functionality. There have been several changes to modules between Python 2.7 and Python 3.4+; the official Python documentation for Python 3.4+ is a great starting place to check for any changes in the modules you wish to use. In particular, we’ve seen that
urllib2was removed, and the
Queuemodule was renamed to
As part of this switch we are also migrating from
virtualenv to the more lightweight
venv was introduced as part of the standard library in Python 3. This offers the same type of features as
virtualenv but doesn’t require external dependencies, meaning that it is easier to troubleshoot and support.
We’re presuming that you do Skill development using a Mycroft for Linux install, installed through GitHub. To test your Skill under Python 3.4+, do the following:
- In the terminal, navigate to the directory where you installed
- Next, you will pull down the dev version of
mycroft-coreusing the command
mycroft-core$ git pull origin dev
- Next, you need to run the
dev-setup.shscript by typing
- This will establish your Python 3.4+ environment
- Next, you need to test that your Skill works. Ordinarily, your Skill will be located at
/opt/mycroft/skills. However, if your Skill is not already installed, install it by using the command
mycroft-core$ ./msm/msm install [URL for your Skill’s git repository]
- Test your Skill by ensuring that you trigger every Intent that the Skill has, including any Stop or Pause Intents, and checking that they have the intended result. If an Intent in your Skill is not functioning, then you are likely to hear the phrase An error was encountered in Your Skill Name when the Intent is triggered.
- Use the resources above to examine your code for Python 2.7 -> Python 3.4+ incompatibilities and resolve them.
- You will then need to resubmit your Skill to the 18.02 branch. Read more here about Skill Branching.
If you encounter a Skill from another Skill Author that does not function correctly under Python 3.4+, then we recommend that you create an Issue in GitHub under that Skill’s repository. We’re very happy to provide assistance that Skill Authors may need in making their Skills function correctly under Python 3.4+, however we do not directly control Community-Developed Skills.
If this all sounds a little confusing, we totally understand. There’s a bit of a learning curve involved in becoming a Mycroft Community Developer, but we’re here to help. You can get assistance in our Mattermost Chat – the Skills channel is going to be the most helpful, or on our Forum.