Well, the simple answer is that it depends on what you are talking about. For self-driving cars that need to react on the near-by traffic, then it must be in the order of micro-seconds. But when you are watching live sports on your TV, you probably do not mind if the footage is half-a-minute old.

At I-CONIC, we are often talking about real-time, instant, immediate, or at-video-rate processing of video into 3D products. In reality, the latency depends on the entire system, and such a system may include delays on-board the drones, on the downlink, on delivery to processor, processing time and finally dissemination to the visualization system.

So, “real-time” might be too ambiguous, it is probably more adequate to define different types of usage of the derived data. Let us try a definition for the usage of 3D data products from a drone flight:

  1. On-the-fly. Delivery, processing, and visualization of 3D processed data is performed in a way that the user of the data can influence how the drone is operated and how the data collection is done, and/or how other types of work in the field are conducted while the drone is flying. Then latency has to be less than a handful of seconds (< 5-10 seconds).
  2. In-the-field. Delivery, processing, and visualization of 3D processed data is performed in a way that the user can check the data collection before the drone operator leaves the field. Then latency has to be less than a handful of minutes. (< 15 minutes).
  3. In-the-office. When the latency is more than 15 minutes, the user cannot really influence drone flight, data collection or immediate field work. Having said that, there is still an enormous difference for many applications if the user gets the processed data after two hours or has to wait until the next day.

When it comes to the photogrammetric processing into 3D models, category #3 is industry standard, with many excellent products on the market. Focus is more on accuracy than on low latency. Some state-of-the-art providers can certainly deliver in category #2.

I-CONIC’s focus is fast delivery based on 3D processing using GPUs. In an on-going project together with consultant company Tyréns, we are developing a complete solution from drone to visualization in category #1. The application is remote monitoring and mapping of landslides using 3D models and derived products.