REDUCTION TO PRACTICE AND RAPID PROTOTYPING -- In today's markets, it is sometimes not enough to just develop IP in the form of patents only. Therefore, we strive to reduce to practice all of our patents. Below are two prime examples. The system below is a prototype of what today is termed "multi-sensor fusion".
The prototype system below on the Volvo S-80 was developed on multiple processors. The processors we chose were the Analog Devices Black Fin BF537 and the Crosscore hardware development kits. We also selected Green Hills Multi and the Integrity OS. The goals were to demonstrate to the automotive industry that multi-sensor fusion could be implemented on a processor the industry was currently utilizing.
The algorithm development was done using MatLab and Simulink. Once verified, everything was ported to C and compiled to the 537 processor using the GHS Multi tools.
Driving demonstrations of the systems we developed were conducted here in Washington State, Michigan, Germany, England and Holland. Click here for videos and still images of the prototype systems.
This operating prototype was developed as the last in a series of prototypes that started in mid 2001 and ended in early 2008. This system generally represents in practice the following key patent families:
- 6,629,033 - Open Communications system -- Here we implemented inter-processor communications via the SPI bus across multiple processors. These abstracted communications carried priority and security labels for secure preemptive processing. This architecture gives the applications the ability to dynamically prioritize processing needs based on system rules.
- 7,146,260 - Dynamic Configuration System -- The prototype demonstrated capabilities of a smart abstraction including device management, data management and configuration management. As noted above, the apparatus had two ranging devices, a radar and lidar. Typically these devices are tightly coupled and tightly integrated during the design and manufacturing process. This system, using a device, data, and configuration managers, was able to demonstrate the discovery and secure connection to a new sensor on the fly -- also known as "plug and play".
- 7,178,049 - Secure Java Virtual Machine -- again with focus on the smart abstraction, this system further demonstrates additional services. These include: message management, critical data management, data logging, and task management.
- 6,615,137 - Method for Transferring Information Between Vehicles -- and 6,792,351 -- Multi Vehicle Communication. In the prototype shown above, note the blue box the embedded prototype is mounted on, this is an 802.11 packet transceiver used to communicate with other local traffic and infrastructure. The messages sent included the test vehicle's kinematic state, and objects detected. The messages received were from other test vehicles as well as stationary roadside transmitters.
- 6,771,208 - Multi-Sensor System -- In the apparatus attached to the hood, two sensors, in this case a lidar and radar, were mounted on a common substrate with a known dimensional relationship. The embedded controller uses the offset information to correct all measurements to a common Cartesian reference frame.
SENSOR FUSION TOOLS
- 8,417,490 - System and Method for Modeling Advanced Automotive Safety Systems, and System and Method for the Configuration of an Automotive Vehicle with Modeled Sensors. These two applications ('608 now allowed) cover all aspects of what will become the driverless vehicle of the future. Our approach was to develop what we termed SaCore, or "situational awareness core", ultimately arriving at the driving prototype above, concurrently with a set of tools based on the same core. The goals were to allow developers to model sensors, model a vehicle with the modeled sensors selected, run the model in a synthetic environment, and generate results one could duplicate in a driving version. This would result in a a much lower cost and schedule during the development cycle.
- 7,337,650 - 7,168,448 - 8,001,860 -- Dynamic Bore Sight -- As a follow on to our '208 family, the '650 family deals with low cost automated methods for aligning and re-aligning sensor apertures to the vehicle reference frame. The approach is to utilize micro-inertials co-located with the sensor, and use a 6-DOF gyro (not found in all vehicles) to provide a reference. if the sensor is out of alignment, the difference in forces are measurable and can be applied to the sensor reports. In a multi sensor system, operating on a uniform grid will be inevitable. The patent family remains in continuation with a priority date of early 2004.
As a result of these patents, Medius was recognized by NASA's Office of Logical Design and asked to present its solution for dynamic sensor alignment, this was a joint presentation with BAE Systems and Celoxica. Click here for the presentation and here for the whitepaper.