Skip to main content

Why we need compile-time reflection in C++1y

Programs need data. That's a no brainer. Programs are only as good as the data you provide them. Based on what kind of data is consumed, programs can be divided into two broad categories: (1) those that operate on regular data (a file), and (2) those that operate on other programs. The first kind of programs are abundant. Your browser, for instance, is showing you this page--its data. The second kind of programs are more interesting and they are called meta-programs.

Meta-programs need data too. As with the other programs, meta-programs are only as good as the data you provide them. So what do we feed them? ... Well, In C++, more important than 'what' is 'when'. (remember Morpheus?) A C++ program is just a sequence of bits the compiler is trying to understand. So, while the compiler is trying to make sense of your program, most of it gets translated (to assembly) but some of it gets executed. Quite intriguing! We're talking about compile-time meta-programming.

Coming back to the 'what'. We want to be able to feed whatever is available at compile-time: types, members, functions, arguments, namespaces, line numbers, file names, all are a fair game. Less obvious things are relationships among types: convertibility, parent/child, base/derived, container/iterator, friends, and more.

A C++ compiler already has this information but it is not in a form a meta-program can use. So we're in a soup, where we can run programs (at compile-time) but there is no data! So the next question is 'how' do we make the data available to our meta-programs? And that brings me to what I like to call the Curiously Recurring Template Meta-Programming  (CRTMP) pattern.

Curiously Recurring Template Meta-Programming Pattern

The idea is rather general and many have done it successfully before: Make data available to meta-programs without offending the compiler and do something interesting with it.

Let's look at who are the subjects (players) in this pattern. (1) the compiler, (2) the meta-program, and last but not the least is (3) the programmer itself because machines haven't taken over yet and humans still write most of the programs as of today.

The compile-time data must make sense to all three above. Today, C++ programmers, because we don't mind pain, create that data in a form that is understood by the former two. The prime examples are the traits idiom, the type_traits library, and sometimes code generators that parse C++ files and spit out relationships between classes.  For example, LEESA's gen-meta.py script generates typelists (Boost MPL vectors) for classes that contain other classes (think XML data-binding). Effectively it builds a compile-time tree of the XML node types.

When things are not auto generated, we make it palatable to the fellow programmers using macros. To many, macros are as obnoxious as the data they hide/generate but lets move on. There are many examples of super-charged too: Boost SIMD, pre-variadic Boost MPL, smart enumerations, and many more. When macros are used in a clever way (abused!) they really do look like magic. I got a first-hand experience of that while developing the RefleX library.

RefleX is a compile-time reflection-based type modeling in C++ for DDS Topics. It is open-source but you need the RTI Connext DDS to play with it. It essentially transforms your native C/C++ type into a serializable type representation called a TypeObject and marshals your data in what is called a DynamicData object. Note that both, type and data are serialized. There are systems--perhaps many we owe our modern life to--that need to distribute types and data over the network for discovery, interoperability, compatibility, and for other reasons.

Here's an example:


The RTI_ADAPT_STRUCT macro expands to about 120 lines of C++ code, which is primarily reflection information about ShapeType and it can be used at compile-time. It is based on the BOOST_FUSION_ADAPT_STRUCT macro. The macro opens the guts of the specified type to the RefleX library. The meta-programs in RefleX use this "data" to do their business. The reflection information includes member types, member names, enumerations, and other ornaments such as a "key". The point is that the same CRTMP pattern is used to "export" information about a native C++ type.

So, the last two open-source C++ libraries I wrote use the CRTMP pattern: In one, "data" is generated using a Python script and in the other using a macro. CRTMP makes C++ libraries remarkably powerful. The reality is there is nothing novel about it. It is seen everywhere.

The natural step in evolution of a idiom/pattern is first-class language support.  If something is so prevalent, the language itself should absorb it eliminate the crud involved developing and writing CRTMP-based libraries.

That brings us to the main point of this post: Compile-time Reflection. We need it. Period. It's a natural step of evolution from where C++ is now. When available, it will make vast amount of compile-time data available to C++ meta-programs. They will run faster, look nicer, and they will knock your socks off! It is mind boggling what has been achieved using template and preprocessor meta-programming. Compile-time reflection will push it two notches up. So stay tuned for C++1y.

Comments

Amber said…
Completely resonates.

I've been using CRTMP to implement JSON serialization and Odata services. It would be soo nice to have language support.

I suppose it'll be a quick vote once Modules become reality. That's the big fish to catch, IMO
Alan Abraham said…
Acetech who have been one of the leading software development company in Delhi giving creative custom software development to meet interesting business challenges for the absolute most distinguished organizations and associations in the country.
Marco Alesiani said…
Totally agree with the need for reflection in C++1y or anytime soon. First class support is indeed required due to compatibility and a bunch of other reasons more strictly compilers-related.

I've been finding myself in the need of a good reflection mechanism many times by now and although I never quite dived into it, I'll definitely take a look at the resources above. Thanks for posting this.
Don Smith said…
Gosh this stuff is soooo over my head! Hate being a noob! lol
Brenda Akers said…
That's really a great information you have shared. I appreciate your efforts, Keep it up.
awais kamran said…
Can someone guide me regarding any c++ in urdu tutorial
Tekslate said…
Wow really good information shared about C++.
Good work Keep it up
John Williams said…
Found your blog. Its really have good information on C++ programming. Really liked and appreciate it. Thank you for all the information.
Peter Carter said…

Great information here. Thank you for sharing. If you are ever in a need of colorado
emergency
Services we are Aurora Locksmith Services. You can find us at auroralocksmithco.com.com or call us at: (720) 220-4851.
Subscribe to my blog if you are a beginner and want to improve your programming skills.
http://khgamujtaba.blogspot.com
Jimmy Hazard said…
This comment has been removed by the author.
P Archan said…
Welcome to C++ Tutorials. The objective of these tutorials is to provide good understanding on C++ Programming Language. In these tutorials, we will Read more
Check this siteTeKslate for indepth C++ training.
Go here if you’re looking for information on C++ Training.
Jimmy Hazard said…
This comment has been removed by the author.
Jimmy Hazard said…
This comment has been removed by the author.
Peter Carter said…
This comment has been removed by the author.
Jimmy Hazard said…

Very interesting. If you have paid the penalty abatement
, we can get them back for you. Head over to refundproject.com or call us at: 888-659-0588.
Peter Carter said…
This comment has been removed by the author.
Peter Carter said…

Great information here. Thank you for sharing. If you are ever in a need of
locksmith denver
Services we are Aurora Locksmith Services. You can find us at auroralocksmithco.com.com or call us at: (720) 220-4851.
Jimmy Hazard said…

Very interesting. If you have paid the penalty abatement
, we can get them back for you. Head over to refundproject.com or call us at: 888-659-0588.
Jimmy Hazard said…

Very interesting. If you have paid the IRS Penalties, we can get them back for you. Head over to http://refundproject.com or call us at: 888-659-0588.
Vardan Soni said…
"Informative article. Intellipaat provide online training on different different Technology . It also provide job assistance, 24 hours support, Lifetime accaess.
"
manho valentine said…
Welcome to C++ Tutorials. The objective of these tutorials is to provide good understanding on C++ Programming Language. In these tutorials, we will
gclub

Popular posts from this blog

Multi-dimensional arrays in C++11

What new can be said about multi-dimensional arrays in C++? As it turns out, quite a bit! With the advent of C++11, we get new standard library class std::array. We also get new language features, such as template aliases and variadic templates. So I'll talk about interesting ways in which they come together.

It all started with a simple question of how to define a multi-dimensional std::array. It is a great example of deceptively simple things. Are the following the two arrays identical except that one is native and the other one is std::array?

int native[3][4];
std::array<std::array<int, 3>, 4> arr;

No! They are not. In fact, arr is more like an int[4][3]. Note the difference in the array subscripts. The native array is an array of 3 elements where every element is itself an array of 4 integers. 3 rows and 4 columns. If you want a std::array with the same layout, what you really need is:

std::array<std::array<int, 4>, 3> arr;

That's quite annoying for two r…

Folding Monadic Functions

In the previous two blog posts (Understanding Fold Expressions and Folding Functions) we looked at the basic usage of C++17 fold expressions and how simple functions can be folded to create a composite one. We’ll continue our stride and see how "embellished" functions may be composed in fold expressions.

First, let me define what I mean by embellished functions. Instead of just returning a simple value, these functions are going to return a generic container of the desired value. The choice of container is very broad but not arbitrary. There are some constraints on the container and once you select a generic container, all functions must return values of the same container. Let's begin with std::vector.
// Hide the allocator template argument of std::vector. // It causes problems and is irrelevant here. template <class T> struct Vector : std::vector<T> {}; struct Continent { }; struct Country { }; struct State { }; struct City { }; auto get_countries…

Covariance and Contravariance in C++ Standard Library

Covariance and Contravariance are concepts that come up often as you go deeper into generic programming. While designing a language that supports parametric polymorphism (e.g., templates in C++, generics in Java, C#), the language designer has a choice between Invariance, Covariance, and Contravariance when dealing with generic types. C++'s choice is "invariance". Let's look at an example.
struct Vehicle {}; struct Car : Vehicle {}; std::vector<Vehicle *> vehicles; std::vector<Car *> cars; vehicles = cars; // Does not compile The above program does not compile because C++ templates are invariant. Of course, each time a C++ template is instantiated, the compiler creates a brand new type that uniquely represents that instantiation. Any other type to the same template creates another unique type that has nothing to do with the earlier one. Any two unrelated user-defined types in C++ can't be assigned to each-other by default. You have to provide a c…