In this post, Tangram Flex Engineer Andrew Allgeier introduces practical solutions to overcoming data transfer obstacles in component-based engineering. Andrew provides expertise in open architecture and cross-domain communications in his work on the Tangram Flex team and is influential in developing Generic Component Software Interface solutions in Tangram Pro™.

We’ve talked about the benefits of component-based software engineering before: testing is easier and more focused, systems are more secure, interoperability is improved, and the components themselves are more maintainable. It’s easy to understand why this approach is valuable, but as we all know there’s no single magic solution to engineering problems. In this post we’re getting technical about the challenges of developing component-based software and practical solutions to them. If you’re just starting out, take a look at our article about component-based design and recent blog posts for an introduction to component based engineering.

Let’s Get Technical — The Data Movement Problem

Component-based systems are lauded as the solution for slow, tedious, and expensive engineering lifecycle problems. On paper it sounds simple enough, but communication and data transfer between components presents obstacles of its own.

When developing component-based software, the ability to move data between components is of critical importance. This problem is generally easy to overcome when simple data ontologies represent the data moving between components. As the data ontologies get more complicated however, as seen in mission critical systems and modern avionics, this problem begins to take on a life of its own. Overcoming the data movement problem can take days and weeks of engineering time away from solving the problem the component based system was originally intended to solve.

Data transfer can be challenging. Even the simplest data sets require that objects have an in-memory representation that can be accessed and manipulated by the application. These objects represent the data, like data coming from sensors, that we want to communicate to other components. The way data structures are stored in memory in modern computing is not always ripe for transmission as the data can be spread all over the application’s memory space. To successfully send data, these structures need to be placed into a contiguous block of memory that can be copied “on the wire” in a process known as serialization. This is only half the battle as the contiguous data read on the receiving side will then need to be unpacked into an in-memory representation that the receiving application can access in a process known as deserialization.

The “simplified” scenario above ignores the entire data transmission process. Data transmission is the physical sending of data from one end point to another and can range from configuring sockets on each component to installing and configuring a data distribution library. This process alone can add weeks to development and integration time. Not to mention that our solution here may work for current system requirements, but what if those requirements change? We are left to re-work the data transmission problem again, adding more time to the schedule before the system can be in the hands of the people who need it most.

Phew that was a lot of work! And wait– we haven’t even started on the business logic our components need to meet the mission level requirements of our system!

Work Smarter, Not Harder

The manual work involved in data transfer and transmission is daunting and at times really boring. What would be great is if we had a simple interface to allow for the easy construction of the requisite data structures or messages and an easy abstraction of the serialization and data transfer of that data. Developing a library with these capabilities is a common strategy for large scale software systems, but these libraries are often specialized to meet the current needs of that system. A simple data ontology update could result in difficult and tedious software updates to library source code followed by regeneration of any assurance testing and validation. What happens if you don’t have the resources to build a library as described? What if you are developing a brand new data ontology either for a new, scalable component based system or even as a hobbyist looking to prototype a new idea? What we really need is a library capable of the abstraction described on the big system, but generic enough to support any use case.

This is a problem we’ve spent a lot of time thinking about at Tangram Flex. A core part of the technology in our CSIP, Tangram Pro™, is our Interface Generation Software that is able to ingest data models and ontologies and generate interface code that component application software can utilize to send and receive data. We started small, focusing on DoD ontologies, and have quickly begun to expand the interface generation capability to support more use cases.

Generic is Better

We call the code generated for sending and receiving data between software components a Component Software Interface, or CSI. Recognizing the value of a library that can support a multitude of use cases, we’ve worked to generalize the generation of interface libraries to produce a Generic CSI. The Generic CSI provides a clean application interface consisting of getters and setters with names based on the field of the provided data structures in the ontology, a variety of serialization and deserialization mechanisms such binary buffer and XML, and a transport layer capable of using variation data distribution libraries to send and receive the data. This level of data abstraction is possible because though the data models may be very different and serve unique needs of their application, all of the elements are at their most basic represented as primitive types such as integers, floating point numbers, and strings. By interacting with these primitives we are able to provide internally re-usable interfaces to serialization/deserialization libraries. Once data has been packed and ready to transmit, a clean wrapper around the supported data distribution libraries provides a consistent interface to transmit and receive data.

That was a lot. The TL;DR? A Generic CSI addresses the data movement problem and saves a lot of painful engineering effort that can be spent on more productive and interesting work.

Confidence is Built In

Software assurance should be a consideration at every stage of software development — never an afterthought. With that in mind, the Generic CSI generated with Tangram Pro™ comes with assurance tools built right in. Our recent work has included use of static analysis tools capable of identifying any undefined behaviors such as overflows, null pointer dereferences, and buffer errors. When we need more in-depth analysis of the generated code, the CSI build process can run tools like Infer and scan-build that to monitor the build process and identify potential vulnerabilities.

The automatic generation of a CSI capable of handling all the data flows between components frees up time for the systems engineering team to focus on defining the correct data ontology for their unique problem set. Developers are now able to focus on implementing and testing the proper “business logic” of their components. This focus on the software doing the right thing will help teams (and hobbyists) rapidly develop and integrate complex systems.

Tangram Flex simplifies software integration for mission-critical defense systems. Every system is unique, but the mission is clear: the people on the ground need dependable, adaptable equipment to get the job done. At Tangram Flex, we understand the challenges of security, speed, and safety. Our team combines engineering expertise with our Component Software Integration Platform (CSIP), Tangram Pro™, to arm engineers with customized toolkits for meeting mission needs.

Tangram Flex is headquartered in Dayton, Ohio. Our staff has experience from DoD, Fortune 500 companies, and innovative software startups. We are dedicated to walking alongside our customers to keep pace with changes in technology. Get in touch: hello@tangramflex.com.