Russell Keith-Magee – A tale of two cellphones: Python on Android and iOS – PyCon 2016

(moderator) As a former mobile developer myself, I’ve been following Russell’s work quite closely and I think you are all going to be very excited by what you see Please make welcome Russell Keith-Magee [applause] (Russell Keith-Magee) Well thank you very much, Chris As you said, my name is Russell Keith-Magee I’d like to welcome the captioners to the semi-finals of the world speed transcription championships [laughter] If you have heard of me before and it’s not because of my reputation amongst the reputation of fear amongst captioners, it’s probably because of my work on the django project I’ve been a core team member on django for almost 11 years and I was president of the DSF from 2011 to the end of 2015 Django’s been around for just short of 11 years and django and the web are still very important platforms They’re not going anywhere in a hurry But over those last 11 years, we’ve seen the emergence of a new type of platform; the mobile platform Delivery of web content for a mobile form factor is one approach for this, that’s and it’s something that has evolved over time as best practice But mobile platforms also deliver a different way to deliver content as native applications Now, it’s no secret the mobile device space is dominated by two major players; Android and iOS But unfortunately, Python hasn’t historically had a really good story for either Android or iOS If you wanted to write applications and mobile applications for iOS you either had to learn objective-c or more recently Swift And then to write your companion Android app, you had to learn Java and a completely different set of API’s for Android Now, there are a couple of other options that are out there using other languages, C#, JavaScript, and even Ruby for varying degrees of cross-platform and native But Python was, for the most part, absent It wasn’t completely absent Kivy has been working in this space for a couple of years But those tools were very much wrapped up in the Kivy toolchain rather than being something generic that you could use without Kivy Over the last two years or so, I’ve been working to rectify this situation The purpose of this talk is to show you what’s currently possible and give you a dive into how some of the pieces work to deliver Python on mobile experience Now, the first thing that I want to address is exactly what I mean when I say mobile Python Other people may and do have very different interpretations of this and they’re all completely valid But I have a very specific definition which rules out certain options When I say mobile Python I’m talking about the development of a native application delivered as a native application using native system APIs Let’s break that apart I’m talking about native applications I’m not talking about HTML 5 apps that are delivered using Python I’m not talking about a native shim that displays a web canvas that then renders content generated by a client-side web browser I’m talking about an app that you start the same way you start any other app on your device that behaves the same way as a natively developed app behaves I’m also talking about apps that are delivered using the normal platform mechanism, downloadable as a single bundle from an official app store Now I know you can jailbreak devices and those approaches open up all sorts of options But any ecosystem that relies on jailbreaking won’t ever really make it to the mainstream You can also do some really nifty things with system services on Android and treat the Android platform as a low-power UNIX server That’s not what I’m targeting It’s a very interesting approach, but it’s not what I’m targeting I’m talking about developing and deploying apps with a potential for mainstream adoption Essentially, the user, end user should not know or care if the app they’re using was written in Python any more than the users of Instagram and Disqus care that the website was written using django if they need to know or care, then in my opinion, you have failed Lastly, the apps use native system APIs The app should look like native apps, feel like native apps, and provide access to any native service the platform provides Using Python shouldn’t be a constraint on your options in terms of what you can achieve Okay, so given that definition of mobile Python, what is currently possible? Well, the TL;DR version is, you can write a native iOS application, you can write a native Android application, and you can write a cross-platform application Okay, so let’s have a look at those Start with iOS We’re going to start with iOS because it’s actually the easier of the two It’s easier because the toolchain that you use to develop native iOS apps is Clang, the same C Compiler that you use to compile CPython on a Mac Now, this doesn’t mean that iOS is just a POSIX It isn’t quite that simple but it’s pretty close Close enough that with a few patches and disabling a couple of modules in the standard library, you can compile CPython 3.4 or 3.5 sources so that they will run on the iOS simulator or on an ARMv6, ARMv7, ARMv7s, or ARM64 processor which covers pretty much every iOS device

that has been sold over the last five years A patch to do just this has been submitted for inclusion in the CPython itself It’s issue 23670 on Roundup Python Bug Tracker There are some design decisions that need to be ratified There’s an issue with the build system that was introduced in CPython 3.5 that needs to be addressed But ultimately, merging this patch is really on waiting on attention from a champion within the core team If that was done, iOS could be a fully-fledged platform supported by CPython Now, I did talk about this at the language summit on Saturday and there are some other concerns that exist That’s going to be an ongoing discussion for some time But you can build it outside of being in the official tree Okay, so how does it work in practice? You don’t have a command line in which you can type Python minus in my app So how do you use CPython on iOS? Well, you treat it as an embedded application You write a very small C stub with a mainline C file that starts the CPython interpreter as an embedded library And that starts the iOS and then starts the iOS event loop Now, I could dig through all this in detail but honestly there’s really no point Firstly, because it’s pretty much a vanilla usage of the embedded CPython API which the CPython docs actually do really well But also because it’s frankly just easier to template it Python iOS template is a Cookiecutter pattern that you can use to generate an entire ready to run iOS project You run Cookiecutter over that template, download the precompiled iOS libraries, drop them onto a, drop into the project Full instructions on the GitHub pages link there and you’re off to the races with a fully functioning Python instance that’s running on iOS So once you’ve got that stub project you’ll actually want to do something You could start by writing a standard hello world but that’s not going to be very enlightening because it’s going to just output to standard out Which you can see if you plug into Xcode but it is invisible on a standalone iOS device What you really want to do is invoke some native iOS APIs So how do you do that? Well iOS native libraries are provided in Objective-C which is a qwerky language but the good news is that it’s actually really compatible with Python Quick aside, Apple, threw a little bit of a spanner in the works a couple years ago with the announcement of a new language called Swift The good news is that this doesn’t appear to be changing anything from Python’s perspective Swift does look really interesting but to my read it’s ultimately just a nice set of syntactic sugars and some type safety around the Objective-C runtime, replacing the qwerks in Objective-C syntax with some dynamic conventions that would be familiar to you as a Python developer There also aren’t any signs that Objective-C is going away And even if it did, Swift is built on top of the same Clang toolchain So they will almost certainly be a way in, it’s just a question of what that way is Okay, so back to Objective-C Here’s some sample Objective-C code If you’ve never seen Objective-C before, it’s just lots of line noise But what it’s doing is constructing a new NSURL object, the Objective-C representation of a URL What we’re saying is we want to pass the URL with string message, the string to the NSURL constructor and get back an NSURL object Okay, so how do we do that in Python? Well the thing is Objective-C is designed as a really thin syntactic wrapper over raw C The qwerky Objective-C syntax is literally a translation to raw C API calls So let’s just roll this out a little bit more We’ll replace the at syntax of @ with a slightly more lower level API So what we gonna do here is allocate a string, NSString alloc, pass a message in it with characters That is not an Objective-C string It is a raw C string of length 16 that will give us an Objective-C NS string which we can then pass into our constructor That series of calls can be translated into this raw C We get a class, we call Objective-C getClass the class called NSString We ask for the alloc selector We pass that selector to the NSString object and we get back an ID for a string class We can then pass, we asked for, initWithCharacters:length message and we pass that message with the variables and 16 in to get the string back And then we do the same thing with the underlying NSURL What we get back is at the end, ID URL, a pointer to an Objective-C object Okay, so that’s raw C How do we get from that to Python? Well part of the CPython standard library is a package called ctypes

Ctypes is a library that exploits the fact that the C calling convention is a very well defined process and can use that to invoke C libraries directly rather than having to write a C module to access functionality in that C library Using ctypes, all you have to do is reference which library contains the code object you want then describe the prototype of the method you want to invoke and you can invoke that method directly So taking the standard libc library here, we could load libc which is the standard C library on Posix’ boxes You can tell that library that the function strchr takes a character pointer and a character and returns a character pointer So what it’s doing is looking for the character D in the string ABCDEF and it will return the string going from that point, returns DEF That code will run without compiling any binary whatsoever You don’t need a C-compiler It just runs as raw Python code operating over the ctype’s lab So if you can invoke C libraries directly, that means you can invoke Objective-C directly Take that very first call to NSString We can load the Objective-C library, tell that library that Objective-C getClass takes a character pointer and returns a generic object pointer Invoke it, and we get back a pointer to the NSString class that we can then use in subsequent message passing calls Okay, that’s neat, but how does that help? Well Python is also a very dynamic language and most aspects of the language at runtime can be configured and overwritten including basic things like attribute access So if you define a class with a dunder getattr method, any attempt to access an attribute on that object will be turned into a call to that dunder getattr method Similarly, for attempts to set an attribute dunder setattr We can then use that to our advantage If you’ve got an Objective-C instance and someone invokes dot something on that instance, I can use that as a trigger to do a lookup on an Objective-C API, find a selector that matches, and retrieve an attribute with that selector Python also allows you to intercept the process of invoking a function If a class implements the dunder call method, it will be used if someone tries to invoke that object as a function Again, we can use this as a trigger to invoke an underlying Objective-C method Lastly, Python gives you the power to control the process of object construction itself Everyone knows about dunder init, that’s the object constructor But there’s also dunder new which can be used to control the process of instantiating instances itself We can use this to bind to a specific Objective-C instance or class Put all that together, and what you get is Rubicon, a library that gives you transparent Python site access to the entire Objective-C runtime Any published iOS or for that matter, OSX, or next step API, is now fair game Now, I should also point out that Rubicon isn’t alone in this space There is also a PyObject-C and PyObjus However, PyObject-C targets desktop APIs, won’t work on mobile without major modification And PyObjus uses Siphon so there’s an easier compiled component Rubicon is pure Python and sufficiently high level that it will work on both desktop and mobile Using Rubicon, accessing Objective-C becomes as simple as loading the foundation libraries, getting a representation of the NSURL class, and then invoking a method on it with the very slight conceit that every colon in a selector is replaced by an underscore There’s no background knowledge of NSURL that’s encoded in the Rubicon library here It’s being dynamically generated based upon the fact we know the name of the class and with the attributes that have been requested You can even extend Objective-C classes in Python space, declare methods that need to be exposed in the Objective-C environment You just need to decorate them to let the type construction process know that the method needs to be registered with that Objective-C runtime This is very useful when you’re defining callbacks But wait a minute, Objective-C is an extension of C C is a type language so how do you represent typing information? Well, you use Python 3’s type annotations You’ll notice that the pokeWithValue method in that slide has an argument value This is an annotated, sorry, and an argument v This is annotated with a type description of int which the type generation process can then use to drive the registration and say I’m looking for an integer argument It also says that it returns none So it’s a void method But if the callback needs to return a particular type or a particular object class you can declare that and it will be used Now to be clear, this is not a custom language extension or a custom Python extension This is simply legal Python 3 And it has been since Python 3.0 This is just a slightly off piste usage of that particular type annotation system From here, it’s just a matter of define the right classes and invoking the right iOS methods But you can use any textbook iOS method,

or iOS textbook, to guide that process Stick that code into a cookiecutter template and you’ve got a working, pure Python, completely native, iOS app All that’s left is to submit it to the App Store and well, profit, I guess So that’s iOS, what about Android? Well, on Android the native compilation toolchain is Java, although based upon recent U.S court cases, no it isn’t It’s not the Sun Java, Sun Oracle JVM though It’s a Java bytecode that is then compiled to system native format you can still call system native code though Java provides JNI, the Java Native Interface, which allows you to call compiled code, compiled C code, from within Java So we can just compile CPython again and link it as an embedded library, right? Well, yes and no Yes, you can do this There is a separate Roundup ticket 23496 tracking a patch for CPython that does exactly this There is probably a slightly more mature version of that patch with a slightly easy-to-use toolchain, that Savied, the guy, has got a fork that works for Android devices And this general approach is the way that Kivy actually does provide Android support However, there is a very big caveat on this On Android you can’t use the ctypes trick because the native libraries aren’t exposed as C functions, they’re exposed as Java In order to have native widgets and access native APIs, you need to bridge between CPython and Java using JNI Now, if you are using a desktop version of Java, and Suns, you know the Oracle Sun Java JVM, that doesn’t have any serious constraints It’s fast, it’s responsive, and it works really quickly But on Android there’s a hard limit of two thousand object references, two thousand individual JNI references That’s hard coded into the kernel And on some devices, it’s even lower It’s not something you can find out until all the sudden you discover your kernel doesn’t have enough references and you need a reference for every data type, every method name, every object instance, every variable that you reference To start Hello World on Android and fully instantiate all the classes that are involved, you need almost 4,000 references So you have to do a lot of reference caching and looking up, and look up and allocating and so on And that means there’s lots of repeated lookups and it’s slow to boot Slow enough that to my mind it probably isn’t viable Kivy does do it that way but Kivy’s approach actually lends it to not having this much JNI interface Okay, so this is Jython This is Java so we can just use Jython, right? Well, no, it’s not that simple First off, Jython only hit Python 2.7 support last year So out of the box you are missing a whole bunch of stuff That may not be a major problem but it’s at least worth considering I’m sure they will eventually get the Python 3 support it’s just not there today Secondly, Jython actually won’t compile for Android It turns out that Java on Android really isn’t Java The Java standard library on Android, Java standard library on Android doesn’t contain a big chunk of java.nio, all the stuff for dealing with files, directories, and file attributes in a platform-independent way Strangely enough Jython needs most of that It also has a bunch of dependencies, ANTLR, Xerses, Netty, a bunch of others And they won’t compile on Android So lastly what you get with Jython is a fully functioning Python parser and interpreter This is, as you might expect, not a small piece of code And unlike a desktop platform, it’s code that has to be duplicated on every install because there’s no system directory where you can install Python and then have all your applications use that common Python interpreter or there is but it’s not a really easy way to use it for our native app distribution Okay, so is there another way to tackle this? Well yes, there is Let’s take a simple Python function It sings you a little song When you run this program, Python parses the code and it compiles it into bytecode Bytecode is a little, if you havn’t come across it before, it’s a little bit like high level assembler It’s a stack-based machine with basic primitives to pushing onto and popping off of a stack Python even gives you the tools to inspect that bytecode, too The dis module contains a disassemble method that allows you to disassemble any function, any class, any module, whatever, to its block code representation And if you run dis over that sing method, what you get is the raw block code instructions Now each of these is encoded in the pyc file in a binary format That’s the human, this is the human readable interpretation of that code, but hopefully it makes some sort of sense We configure a loop setting the bounds of that loop to be line 47, line 47 to line 50 We load a global symbol called range We load three constants onto the stack, 100, 0, and -1 And then we call the function We invoke a function with three positional arguments That’s range, 100, 0, -1, and so on Okay, that’s nice, but so what? Well let’s do the same thing in Java Here’s the same code We can compile this with Java C and we get a sing.class file

And if we look into that class file, what do we see? You see bytecode Now, okay, it’s different bytecode to Python but it is stack-based And there’s lots of commonalities You are loading and storing variables, you’re getting symbols, pushing constants, invoking methods So if they’re both black code and they’re both stack-based can you convert from one to the other? Yes VOC is a tool written in Python that can convert Python 3.4 bytecode directly to Java 6 class files What’d you end up running, or when you end up running your Apple class file, it’s not a Python class, Python code being interpreted It is indistinguishable from a Java class And that means you can bridge seamlessly between them You can reference Python classes in Java and vice versa There’s a couple of very small caveats on that but for the most part, you just write your Java code but in Python So you take your standard textbook Android API, render them into Python, compile them to Java, and profit (laughter) Here’s a snippet of some Android code setting up a main activity from android.widget import ListView We then define a main activity that extends Android on app.Activity and again that’s Python 3 legal syntax We define an onCreate method that takes self because it’s Python, and a state argument that is an Android.os.Bundle And that method returns a void We invoke supers onCreate method, passing in that state We then log something — log MyApp, Create new app — and then we create a list view, a new instance of ListView That’s all Python code and it spits out to a class file that you can then run through the whole Android toolchain to dexa and turn it into a runnable binary So as with iOS, there’s a bunch of Android housekeeping that’s needed around the outside of that to actually get it running and there’s a cookiecutter for that, too You run CookieCutter over your template, download the precompiled Python Android libraries, and it will even manage compiling the Python code to Java and packaging up as it needs to Again, full instructions on that GitHub page Okay, so, so far, we’ve been able to write a native iOS application using iOS APIs in Python We’ve been able to write a native Android application using Android APIs in Python Can we do this cross platform? Can we write our UI code once and have it render on both platforms? Yes One approach, one I’ve mentioned a couple of times already is Kivy Kivy doesn’t use native widgets, though It treats the fine surface as a canvas on which it can draw And it draws on that canvas a button When you place a button on the screen, it’s a Kivy button, not an iOS button or an Android button So it doesn’t look like an Android button, doesn’t look like a Kivy button Sorry, it doesn’t look like an Android button or an iOS button It looks like Kivy Now, for me, that’s not a satisfactory result It’s okay for applications like games where the visual style of the game is much more important than platform native behavior But if you’re writing a general consumer app, I’d vastly prefer to be looking at widgets that are consistent with a native platform and acquire all of the accessibility capabilities of that native platform rather than a lowest common denominator that’s native nowhere And it’s not just a matter of taste Both Android and iOS have all sorts of affordances built in to deal with accessibility issues Yes, these could be implemented in Kivy but to the best of my knowledge, they’re not If you use native widgets you get all that stuff for free This is where Toga comes in Toga is a cross-platform widget toolkit that uses system native widgets It started on desktop platforms for OSX, Linux, and Windows but I’ve been focusing on mobile recently to provide a compelling reason for someone to give a damn about yet another widget toolkit Toga is a wrapper around system native calls So you have a common interface to generating system native widgets However, you still have all the access to platform internals if you want So if you want you can drop down to a completely native API call whenever you need to So here’s a bit of a taste of what Toga looks like in practice You define an app that extends the base toga.App class Startup method even lets you build the app In this case, we’re going to have a list widget When you swipe to delete an item on the list we’re going to remove an entry and when we pull down to refresh we’re going to call a refresh method Startup method also describes a container that’s going to hold lists We set the main window for the app to be that container The top level container is going to have an action on it, a button up in the top right hand corner on iOS When that button is pressed, we want to show a dialogue And the add input dialog has a container that holds an input widget Now the bit that you can’t see here because it’s just using system defaults is the layout of the container is driven by CSS Oh, I could for example, put extra padding around the end point if I wanted to do that to bring it in from the screen The dialogue has an accept and a cancel action When accepted, Add entry will be invoked

And last, we define some call backs When we show the Add dialogue we can clearly input as well When you add an entry, you get the value of the input and add the new list entry with that text and clear it Remove and refresh follow a similar sort of pattern And that’s it And if you want to see this running action this link is a video to a full version of that app running on iOS, a simulator in the movie But it does work on device as well and on Android And for bonus points almost exactly the same code running as a single page web application Which means Python code running in the browser utilizing a UI declared entirely in Python One or one and a quarter source code bases, three apps, all platform native It’s linked to full source code for all those as well, including the native versions that the Toga applications effectively reduce into it The last piece of the puzzle is making it easy to take this generic Python project and turn it into a working iOS or Android project, and that’s where Briefcase comes in I’ve spoken previously about using CookieCutter to generate projects Briefcase is then a setup tools extension that uses CookieCutter and the project metadata from to generate ready to compile iOS and Android projects How do you use it? Well in your project’s virtual environment, install Briefcase, and you say Python, iOS or Python android Then build the same way you normally would Open the project next code or run an ant build I’d like to expand This also manages the build to do the deploy to device process as well Other tools have demonstrated this sort of thing is at least possible so it’s really just a matter of putting all the pieces together And just quietly, Briefcase also works in OSX and Apple’s TVOS I’d like to expand it to Linux I’ve had a couple of people ask about whether it could work on Windows, maybe even Apple’s watchOS Lots of possibilities there Okay, so what does the future hold? Well there’s still a lot of work to be done I can’t speak for Kivy I’m sure they’ll continue to evolve their APIs and platform I wish them all the best I intend to keep working on Rubicon, VOC, and Toga Collectively, these tools make up what’s called the BeeWare project If anything I’ve said today sounds interesting and you’d like to get involved, I’d appreciate any help There is plenty to do and I’ve got an open offer to mentor anybody who wants to get involved even if it’s your first time contributing to an Open Source project and you don’t think that you’ve got what it takes to be an Open Source contributor because you’re just a beginner programmer, you don’t really know that much yet Yes, you do; yes, you know enough Yes, you can contribute And as an add bonus, thanks to MaxCDN Anybody who does contribute will receive an exclusive BeeWare challenge coin Nice and shiny I’m here to the end of the sprints and I’ll be working on VOC and Togo The BeeWare was hunkered down in booth 313 Unfortunately, I think we are tearing that down now So some people think, look at what I’ve been doing for the last two years and think that I’m tilting at windmills and they’re at least partially right But, I think that as a mobile, as a language community we’ve got a three huge threat on our hands Mobile as a platform is starting to become a really important part of the development landscape And if Python doesn’t have a good story on mobile, we run the risk of being left behind as a language And that would make me really sad because as a community we have built something really great here, a language that’s accessible to newcomers but robust enough for professionals, useful in multiple domains, with an amazing community spirit But the threat is also an opportunity Many of the areas where Python has gained traction, in science, in education These are areas where mobile has the potential to make a huge impact Imagine a world where a scientist can knock out with very little programming experience, can knock out a quick user interface to put in the hands of experimenters to gather information or provide an app so that citizen scientists can log local flora or environmental conditions Imagine a world where you can get kids excited in programming because they can build something that runs on their phone they can show their friends Imagine a django girl style event where students leave the tutorial having not just built a blog but having built a mobile phone app they can use to upload photos and blog entries to that blog In short, it is the best of times, it is the worst of times, it is the age of wisdom, it is the age of foolishness, it is the epoch of belief, it is the epoch of incredulity Like a great Dickens novel, the tale of two cellphones is just getting started If you want to know more, here’s my contact details [applause] (moderator) Right We do have a few minutes for questions If you’ve got questions, please line up at one of the two microphones here and if you could please be relatively quiet during the question asking time so that everybody in the audience can hear the questions and the answers You over here (audience member 1) Can you offer a few examples of apps that have been built using these processes? (Russell Keith-Magee) No, I cannot Unfortunately, like I said, this is very early stages There is definitely evidence The Apple App submission guidelines have made it clear that this approach is possible and there are Kivy apps in the App Store that use the same process that Toga uses However, this is early stage stuff I’m really only sort of just pulling this together in the last couple of weeks to be at the point where it’s really presentable here

So there’s definitely still work to be done And as a result, I have not actually pushed a useful application, partially because Apple will reject apps that appear to be trivial So yes, that’s it — we are still a work in progress (moderator) And one over here? (audience member 2) Hi, it’s really amazing I’m blown away My question is other languages have done this kind of sort of piggybacking on Android and usually they run into sort of memory issues that are either too big or not fast enough Can you comment regarding the performance of these apps? (Russell Keith-Magee) Sure, okay So on Android, the speed performance is slightly less than if you had written it completely native Because there is, like for example, if you’ve got a method that takes an integer, it needs to be boxed to turn it into a Python integer so that it can be processed and then unboxed to turn it back into a Java integer So there is some overhead associated with that However, it’s not huge, you know It’s a manageable level of overhead And it’s also something that with more attention, you actually have enough information in the compilation process to optimize that path If you know this variable is an integer, that’s afloat The only thing you do is multiply and then you return it You have the type of information to not have to box that and just go straight to the mathematical operation So in terms of application size, the Toga Hello World app that I showed just there, that’s about a 6 MB binary when it actually gets deployed So it’s not a, it’s not a huge runtime when it actually gets deployed to fire It’s very similar for iOS (moderator) We will take one more from over here and then that’s all we’ll have time for If you do have questions, you can come up and see Russell afterwards (audience member 3) So great talk, that was very inspiring (Russell Keith-Magee) Thank you (audience member 3) I have a question It would be kind of be magic if it would be possible for — to compile a NumPy on iOS and Android; is that possible? (Russell Keith-Magee) I haven’t looked into it NumPy as I understand, does have a lot of C extensions in it It’s at least in the realms of the possible from what I understand I haven’t played with it at great length, sorry That’s all warranty void where not applicable, etcetera But, you still, like you are writing Python code that compiles to a Java class That Java class could be making JNI calls So there is at least some potential for that to be done Exactly how that works, I don’t know But certainly, if you’re hanging around for the sprints, I’d certainly be interested in seeing exactly what that would look like (audience member 3) I’ll be at the sprints (Russell Keith-Magee) Excellent (moderator) Russell Keith-Magee everyone! (Russell Keith-Magee) Thank you [applause]