Reflection Architectural Pattern

so today we’re going to be talking about two radition ‘el architectural patterns and these are patterns that are big features or even dominant features at the landscape right now in terms of software engineering and will likely become even more dominant in coming years um and the first of them is is a technique known as reflection okay um and reflection is a technology of a set of functionality that is typically built into languages these days so if you look at new languages coming out they typically have significant support for reflection examples of this would certainly include Java and more recent versions but also Scala also Python also c-sharp many many languages you will have heard of be familiar with include support for the sink called reflection and yet reflection is to my knowledge not really talked about in other CS courses maybe you could give me feedback of you have you had this topic of reflection and software discussed in other courts okay so I’m glad covering at them because it’s an important thing to know about not least because it can allow you to to build really interesting tools and one of the areas that it shines the most is actually in combination with aspects um and it allows aspects to kind of discover the structure of the program around them so they can do appropriate things but as many many uses beyond aspects um and really it allows for flexibility um in a program instead of hard-coding a bunch of cases that are dependent on the rest of the structure of the program it allows for discovering that structure at runtime and in doing the right thing based on that structure so let’s suppose you wanted to have an aspect that every time you ran conversing of life with log with log all the parameters the parameter sumption x’ for it you could have this aspect actually ask the Java Runtime hey list to me all the variables in this program and you’d find the ones that are not standard variables but have been added by the user and you could log each of them so it can allow for self-discovery you for program structure and self adaptation of that structure but to give another example suppose that you have classes in your program which are evolving over time but you know that whatever the structure of those classes you’re going to need to be able to persist them to a database in other words write them out if you have an object you want to write it up to the database but over time you know that people are going to be adding variables instance variables in the fields to this class um and yes every time you add a field you could go remember oh I’ve gotta go you know put another right into a different column in the database or whatever but another way that that reflection could work with aspects is you could every time you’re going to write um you’ve changed or excuse me every time that it needs to be output you could discover what the fields are in this class and just write them successively to appropriate named columns in a database so by discovery of the structure of the program you can have a very general program or one that you don’t have to every time update it consistently it will discover it automatically and do the right thing um so it’s very convenient often and the basic way it works is to allow for treatment of the program structure the structure of the program around you as data so you can at runtime or as we’ll see at compile time reason about the structure of the program around you what classes are there what are the fields those classes what methods are there what are the parameters those methods take what are the types around me um and you could do things like call a method based on a string that encodes the name of the appropriate method so you can you could look up in a table should I call foo bar or bass and call the appropriate one based on a string obtained from that table or from user input okay so you know that was one example of a reflection mechanism here are others may need to go quick because they want to I do want to get to inversion of control on discovering names of class field to

match that invoking a method based on a name creating a class instance from a string that includes the name of the class so you don’t say new foo a new instance of class foo you say new of some some class name that’s encoded in a string and depending what that string is maybe you read it in from a config file you create a different class instance of a different class okay um adding new classes at runtime um creating a proxy or mock for a class so so we could ask what are the what are the methods of this class so that we can build a class that supports all those methods and so when we call it instance that class it logs some information checks some information and passes it on to the real cause um in short we can kind of discover aspects of of a program now what are some applications of this well some of them are lists are listed here creation of mocks and stubs and just mention proxies persisting data so another is putting it out marshalling data sending data over over a network you about you have an object over here on this machine and you want to instantiate a comparable object over on this other machine you want to send its data across the wire to do that you need to know the structure the object and one way to do that is to just be always careful that there’s a marshaling method that stays in sync with the structure of the object every time you wani change the other another way is to use reflection you can in debuggers inquire ok what’s the structure of this program so you can say on the command line call this method call that one you can allow for inspectors your program or visualization of a given class etc ok now what sort of tools provide support for this well it turns out it’s very common these days on C++ was an early adopter Java around the same time I think c-sharp um also the popular language in framework that’s quite similar in some ways to Java it has some nice additional and different features Scala and Python are two two big ones but this goes way back it goes back for four decades at some level to the 1970s with small talk and in with Lisp and in fact with Scala and Haskell and a lazy evaluation you can do some things that are kind of similar to to what you get with with reflection but Scala actually has much richer reflection mechanism beyond that um and then aspect-oriented program tends to work with it well ok so when we talk about reflection about discovery of the structure of a program we’re talking about two different types of reflection often one the one I concentrate on is runtime revoke your code is actually running in it asked hey give me a list of all the classes in this program or for this class give me a list of all the methods give me a list of all the fields it can do that at runtime and based on that it will you know write it out to a file the names of the values of each of those fields for a given object so runtime reflection is perhaps the most common case but there is compile time reflection and here during the compilation of a program you can refer to structures in the same program you could say for example that um we want to refer to to this as an instance of so this certain element has to be drawn from the classes of this program and it will actually use that information and transforming it and so Scala macros well I don’t have time to cover them here they they provide a way of transforming the program using almost an extensible language so you can build up language extensions essentially it’s almost like you could add whole statement types to Scala so you have to match you have if you’re four but you could add your own using macros and you could add you know while on until or are do-while or what-have-you once that aren’t already in there you could add just two Scala okay um so just to give you a sense in Java these are some of the things you can do you can um have hey give me a reference to a class not an instance of the class but a reference to to the class so we could call like a static field Ecstatic method on it using class for name so you say hey class dot for name too and it will give you a

reference to the class foo if it exists in your program okay or maybe you ask plasti for name of bart that’s in quotes right maybe it’s a string again you’ve read in from a config file maybe it’s a string the user entered in a debugger and you ask hey give me a reference to that class maybe then you ask for example um get me the methods of that class or get me the type parameters of that class is it a you know array of ArrayList of integer or what what are the type parameters um and you can ask what are its super super types etc um so another thing you could do for example is you could ask for the fields of a class so what fields what information does that elements of this class store so that we know what has to be written out to a file to save away the contents of our code of our program memory at a given time and you can get the the types etc you can get the super classes etc so these are all Java mechanisms for reflection and they’re pretty neat actually one of the most flexible of them is just get annotations annotations were one of those things I was hoping to cover this semester but there just wasn’t time annotations allow you to mark up your program with custom tags that you folks design okay so you’re a developer hmm and you can define your own annotations maybe these are annotations related to your particular stylistic mechanisms maybe their annotations related to um the units of different quantities this is measured in you know meters per second and this is you know feet per minute and you want to keep those separate maybe their annotations with respect to testing or mocking in any case you can take your program and you can mark it up with custom information that’s actually understandable by the program itself or by a tool working on the program and you can then ask hey get me all the annotations you know within a class or get me all the annotations assist you with a method and it will tell you appropriate annotation maybe the annotation is who modified at last or when it was last modified okay so so um you want to make sure you’re operating on methods that are from the latest release and so you can be doing assertions to make sure this is an up-to-date method or something along those lines um really useful um so uh you know within field for example we can get the name of the field the modifiers to type the declaring class and we can get the value of that fields in a given object so given an object we could say hey get me the value of this field in that object maybe the field is called you know um is called you know current and your current value and you get the value and it gives you back to value three and then you can set the value at run time you could say hey for this field set the value and the point is that you can you could do so without hard coding what the field day meant that could be based on a field as specified by the user and debugger or what have you and similarly for a method you can get the name of the method you can get the return type the parameter types and you can in fact call the method with appropriate arguments so think about that in the program you don’t know what method it’s going to call ahead of time but what method it calls might be drawn from looking up in a hash table a certain bit of information you get back the method for these value I call this method for these values I call this method for these values that call this method so you look up at runtime what method should they call and then you actually call it pretty interesting functionality pretty flexible functionality um and very straightforward I don’t think we have time for this but I provided some example code that can for example list out annotations and obtain for example class information here um so if you’re interested in how to how to use these things and what used to them is to create proxies proxies are things that kind of look like they’re doppelgangers for a given class you have a there’s some class and you want to create things that look like it for some reason and

one classic uses mock objects you know you’re going to call these doppelgangers and they’re going to check some things about the variables you’re passing to it whether it’s meaningful their record how many times you’ve called this etc and then they’re going to call X the actual object for example or maybe just return a fixed value or random value so if you’re doing testing mock objects are really useful and in fact you’ll find a whole set of tools out there including je mock and maketo for Java that use mock objects and one of the easiest ways to implement it is with Java reflection so it’s another example of useful things you can do with reflection is implement mocking frameworks or for that matter testing frameworks like j-unit right j-unit looks for annotations and the later versions that match have an annotation indicating that this method is to be tested on older versions look for the names of method which match some string and and would use those as test methods either way benefits from this sort of in at this sort of interface in reflection getting annotations for example getting the names of each method by calling for example get methods and then for each such method getting the name of it that’s how tools like that can be built that operate on Java program okay um and then another thing you can do with proxies is you can for example create a proxy for class and when it call the proxy it secretly goes off and calls calls off to log the information then passes on the call to the actual method or maybe it calls over the network to another machine which actually does the work in any case can be very useful to create proxies and you can do them with you’ve guessed it reflection okay so thanks mocks Stubbs different ways of kind of arm of sort of creating proxies that will help in testing or in building a program really easy to build with with reflection okay um and generally the stupidest Inc shion’s between these fakes return trivial values often random or identical mocks often check certain invariants they check hey am I being called in a meaningful way maybe I should only be called once during initialization have I now been called twice um they’ll check for errors they’ll check the the values being passed to me are these same values and they’ll maybe log things um and and then return values that might involve a simple calculation that that obtains much of the time or just returns a simple value and um and then stubs are often used to talk about kind of proxies for particular methods when that method isn’t yet written so when you’re building a program just like when you’re building a building use scaffolding around it when you’re building a program often use fakes and mocks while things aren’t yet implemented you put them in place and while you haven’t had time to cover it here or in fact in 371 there’s a popular methodology called tracer bullets which has to do with building up a program essentially from stubs so you you define your interfaces for the program what calls what and then you stuffing out so you create stakes kind of little stubs for lots of methods fakes for the whole classes and then over time you fill in more and more of it okay so so you start with kind of fake versions of everything so at least everything compiles and you can in fact place calls they just return trivial things and then over time you expand more and more of this with with actual implementations and the pragmatic programmer series has has some nice comments and tracer bullets that are very very thoughtful and gives some good ideas for how to apply it in programs okay um so reflection now one of the more advanced uses of reflection is to automatically generate code and here um the idea is that we will actually generate at runtime proxies and stubs for a program um or we’ll provide services so we’ll provide for example class browsing or pool management of transaction management and there’s a technique known as generative programming which I would

like to have covered but ran out of time we’re basically on you have a system that will at runtime generate code this is becoming quite more this becoming more common I wouldn’t say it’s incredibly common but you see it in certain contexts so on the Scala Play Framework for example has some generative code for JavaScript so it actually generates JavaScript on the fly based on the structure of your system it can actually generate JavaScript for for your website on another use of this is in if you want to speed up a program so if you discover that that in the program you are calling a given method again and again and again with certain fixed parameters maybe you’re calling it 10,000 times with the first two arguments fixed only varying the third you may want to create a specialized version of that function with those first two arguments set already all the computations involving them already done and all it does is take the third one and operate on it can anyone remind me what if this sound a little bit like in the way I just described it what does it sound like Korean Korean is very similar to this and Eric is very cute just sloppiness so in Korean remember you break up instead of having a function which takes a whole bunch of arguments and performs the computation you break it up into a series of functions or closures so you provide some of the arguments early on and some later we call this ladies and gentlemen staged computation stage not in the sense that it’s baked to be sure but in the sense that it’s a person faces and stages stage computation okay so generative programming can be thought of as a form of staged computation you you recognize now wait a minute we have these certain parameter values fixed and known well ahead of time we have a loop if the loop you know goes over and give gets certain values for the third argument but the first two are always set and this loop runs many many many times and it’s a core core loop of our system so in that case it may be worth creating a specialized version of that code and here the idea is that the system is essentially doing the qurĂ­an for us and in a way that will allow the computations involving those two first elements to be done upfront in ahead of time so let’s suppose for example you have a young a dot product of two vectors x and y and maybe these are large vectors you know each of size a thousand um but you know X ahead of time so maybe you have some for loop and you are performing to you know some for overwise drawing wise from some something and for each of those you’re taking the dot product of a pre-existing X with this Y that’s changing well if you know X ahead of time you may be able to do this much faster because you know for example some of the values of X are 0 and dot product for those who don’t remember if you have to a dot product of two vectors x and y on the way in which you perform them is you have X sub I times y sub I so so here we have a vector X and it has successive elements at at some index I and we have a value vector Y and it has some elements maybe there are thousands of them and it has index I as well referred to and the dot product is just multiplying each success to pair here and summing up the values but if these pairs is a lot of these are zero and others are one we don’t really have to do the work when we see Y of actually doing the multiplications we just notice skip those values of Y because it’s going to be whatever it is here x 0 we don’t need to add that up this is an ad right it’s adding each this times this Plus this times this etc and if it’s 0 Y you can add it in or no we’re adding 0 so if this is sparse vector we might be

able to save half the time with a specialized version of this because if we’re not actually performing most of the computation or not actually performing the multiplications we know we don’t have to because of the value of x so tools like this for staged computation are becoming more common um and sometimes if you think about your programs there’s a huge amount of things that are known ahead of time an example of this would be if even if you have a seed program right there’s lots of cases where you may have printf with some known strings right and all that the whole string is known the the location of the things to be substituted in you know the dollar sign D and the dollar sign s we know exactly what string we’re going to be doing it with respect to all we don’t know maybe is the the particular values you know x y&z here that we’re going to be substituting at right um and if we know the spring ahead of time we’re doing a lot of extra work needlessly if every time we’re interpreting the string saying where does the first value go you’re scanning through it oh it goes here substituted that and then where does the next one go it goes here every time you don’t printf it’s doing every darn time it’s rediscovering the structure of the string every single bloomin time it’s looking through here and finding that structure and substituting in the value and if this is in a loop I mean you’re needlessly doing a tremendous amount of work just to sort of rediscover the structure over time so you could create in principle a specialized version of printf for this string which given these values it just substitutes it in or just appropriate values in a string boom you’re done yeah in a character straight where this is all mapped it figures out how to format as as a it’s an integer string and substitute it in directly so in short going from this to a specialized version of printf is yet another example of staged computation and compilers are becoming more savvy about this in the tools that you can build are becoming more savvy so um there are tools out there which you can build which will allow you to at runtime create a specialized version of dot product or specialized version of printf for particular strings to learn how to do that so that the code can run can run faster generative programming is becoming a fairly routine in certain areas because of the ability to speed things up in stage computation is a really good example of why it’s needed because we often know some things much earlier than we know others now we can manually put in currying to our code and that can speed it up greatly but we also may turn to language tools to do this for us automatically to recognize cases for us okay um okay uh I think I’m going to go on to UM armed to the to the maturity on inversion of control but I’ve tried to give you a glimpse of reflection and reflection is all about self-aware programs programs that can discover their structure and and operate on that structure and do better things based on that structure I will just know that aspects because with aspects you’ve get the aspect invoked even as the structure the program evolves the same aspect can reliably work because it it recognizes cases where you know a certain thing is being called even if it’s with new code code written after the aspect was created it recognizes oh you’re calling a certain thing there and it does the right work there maybe it starts a transaction maybe it persists something to disk maybe it logs a message maybe it it reports something to to the user maybe it visualizes something this is going on while the program’s running and even if the program evolves that aspect is still running it can its robust under evolution the program and therefore ladies and gentlemen the aspect is not going to be able to count on knowledge of the structure of the program ahead of time because the structure of the program is evolving even after the aspect is written it doesn’t it can’t be sure oh the classes are only X Y and Zed or the methods in those classes are only

these are the fields in this class are only those because the aspect is designed to be robust under program evolution it needs to be flexible enough to handle that evolution the program and what it does when it’s invoked as well not only do we need the the aspect to be able to be involved flexibly oh gosh I might crash the computer ok better watch that out um um but it needs to be able to do the right thing with the evolve program and that’s where a reflection can come in because the aspect wakes up oh this thing has been called it’s time to log on what do I log well maybe what I log depends on what fields there are or what parameters are for something and it can look up at runtime hey what fields are there in this you know this class that I’ve just entered via a call and it enumerates those fields and for each of those fields for this object it says what’s its value and it logs that out in short reflection and aspects dovetail really nicely aspects allow for intercepting the program at a certain point in reflection allows for saying what is the structure of this program currently so that I can I can operate on it you know what what what thing do I need to visualize given that I’m here what do I need to visualize given that I’m trying to log things out what particular values do I need to log out that will depend on the structure of the program and that might have changed after the aspect was written the aspect will intercept it and then it will use reflection to find it discover that structure and do the right thing that make sense okay aspects in reflection work together nicely and often they um they really give the greatest power to the aspects more power than it would have otherwise okay reflection some comments on that let us now go ladies and gentlemen