Discover Metavision® Intelligence Suite
5x Award-Winning Event-Based Vision Software Toolkit.
Free to Download, Free to Build, Free to Sell
5X AWARD-WINNING EVENT-BASED VISION SOFTWARE SUITE
95 algorithms, 67 code samples and 11 ready-to-use applications in total so you can either start your discovery or build your Event-Based Vision product. Get started with the most comprehensive Event-Based Vision software toolkit to date.
OPEN SOURCE ARCHITECTURE
Metavision Intelligence Suite is based on an open source architecture, unlocking full interoperability between our software and hardware devices and enabling a fast-growing Event-Based community.
LEADING EVENT-BASED MACHINE LEARNING TOOLKIT
Build your advanced Event-Based ML network leveraging the most performant object detector to date spotlighted at NeurIPS, the largest HD dataset, Event to Video and Video to Event pipelines, Training, Inference, grading features and more.
6 MODULE FAMILIES
With a wide range of computer vision fields covered: Machine Learning, Computer Vision, camera calibration, high-performance applications and more, the tool you are looking for is here.
EXTENSIVE DOCUMENTATION & SUPPORT
With 300+ pages of regularly updated content on docs.prophesee.ai, more than 20 Jupyter notebooks, reference data, extensive guidelines, get a head start on your product development.
GET RESULTS IN MINUTES
We took over 6 years to perfect the largest collection of pre-built pipelines, extensive datasets, code samples, GUI tools and more, so you could get results in minutes.
DOWNLOAD, BUILD, SELL - FOR FREE
Our software toolkit is available on Windows and Ubuntu PC. It is entirely free, without any time constraint. It includes a commercial-grade license, enabling you to design and sell your own Metavision Intelligence-based product, leveraging Prophesee Event-Based Metavision sensing technologies.
APPLICATIONS
DETECTION INFERENCE
Unlock the potential of Event-Based machine learning, with a set of dedicated tools providing everything you need to start execution of Deep Neural Network (DNN) with events. Leverage our pretrained automotive model written in pytorch, and experiment live detection & tracking using our c++ pipeline. Use our comprehensive python library to design your own networks.
Pretrained network trained on a 15h and 23M labels automotive dataset
Live detection and tracking @100Hz
DETECTION TRAINING
Train your own Object Detection application with our ready-to use training framework. Experiment with multiple pre-built Event-Based Tensor representations, and training network topology suited for event-based Data.
4 pre-built tensor representation
Automated HDF5 dataset generation
Comprehensive training toolbox, including Tailor-made preprocessing, DataLoader, NN architecture, visualization tool and more
VIDEO TO EVENT
Bridge frame-based and event-based worlds with our Video to Event Pipeline. Generate synthetic data to augment your dataset, and partially reuse existing references.
Off-the-shelf, ready-to-use Event simulator
GPU-compatible Event simulator seamlessly integrated
EVENT TO VIDEO
Build grayscale images based on events with this neural network. Spare resources by facilitating automatic Event-based data annotation for tasks already solved in Frame-based.
Off-the-shelf, ready-to-use Event to video converter
Leverage existing frame-based neural networks to obtain labels
DETECTION KPI
Evaluate your detection performance with our Object Detection KPI toolkit in line with the latest COCO API.
mAP, mAR and their variants included
OPTICAL FLOW INFERENCE
Predict optical flow from Event-Based data leveraging our pretrained Flow Model, customized data loader and collections of loss function and visualization tools to set up your flow inference pipeline.
Self-supervised Flownet architectures
Lightweight model
OPTICAL FLOW TRAINING
No ground truth? Leverage our self-supervised architecture. Train your Optical Flow application with our custom-built FlowNet training framework. Experiment with 4 pre-built Flow Networks tailor-made for your Event-based data.
Comprehensive flow training toolbox, including different network topologies, various loss functions and visualization mode
GESTURE CLASSIFICATION INFERENCE
Run a live Rock Paper Scissors game with our pre-trained model on live stream or on event-based recordings.
Off-the-shelf, ready-to-use classifier
GESTURE CLASSIFICATION TRAINING
Train your own object classifier with this tailor-made Event-based training pipeline.
Comprehensive training toolbox, including Tailor-made preprocessing, DataLoader, NN architecture, visualization tool and more
VIBRATION MONITORING
Monitor vibration frequencies continuously, remotely, with pixel precision, by tracking the temporal evolution of every pixel in a scene. For each event, the pixel coordinates, the polarity of the change and the exact timestamp are recorded, thus providing a global, continuous understanding of vibration patterns.
From 1Hz to kHz range
1 Pixel Accuracy
SPATTER MONITORING
Track small particles with spatter-like motion. Thanks to the high time resolution and dynamic range of our Event-Based Vision sensor, small particles can be tracked in the most difficult and demanding environment.
Up to 200kHz tracking frequency (5µs time resolution)
>Simultaneous XYT tracking of all particles
HIGH-SPEED COUNTING
Count objects at unprecedented speeds, high accuracy, generating less data and without any motion blur. Objects are counted as they pass through the field of view, triggering each pixel independently as the object goes by.
>1,000 Obj/s. Throughput
>99.5% Accuracy @1,000 Obj/s.
PARTICLE SIZE MONITORING
Control, count and measure the size of objects moving at very high speed in a channel or a conveyor. Get instantaneous quality statistics in your production line, to control your process.
Up to 500,000 pixels/second speed
99.9% Counting precision
OBJECT TRACKING
Track moving objects in the field of view. Leverage the low data-rate and sparse information provided by event-based sensors to track objects with low compute power.
Continuous tracking in time: no more “blind spots” between frame acquisitions
Native segmentation: analyze only motion, ignore the static background
CALIBRATION
Deploy your applications in real-life environments and control all the optical parameters of your event-based systems. Calibrate your cameras and adjust the focus with a suite of pre-built tools. Extend to your specific needs, and connect to standard calibration routines using our algorithmic bricks.
Lens focus assessment
Automatic intrinsics camera calibration
EDGELET TRACKING
Track 3D edges and/or Fiducial markers for your AR/VR application. Benefit from the high temporal resolution of Events to increase accuracy and robustness of your edge tracking application.
Automated 3D object detection with geometrical prior
3D object real-time tracking
OPTICAL FLOW
Rediscover this fundamental computer vision building block, but with an event twist. Understand motion much more efficiently, through continuous pixel-by-pixel tracking and not sequential frame by frame analysis anymore.
17x less power compared to traditional image-based approaches
Get features only on moving objects
XYT VISUALIZATION
Discover the power of time – space continuity for your application by visualizing your data with our XYT viewer.
See between the frames
Zoom in time and understand motion in the scene
DATA RATE VISUALIZATION
Understand the process of event generation over time, visualize data and generate plots with the power of python.
Ready-to-use Python environment
Interface with recordings & live cameras
ULTRA SLOW MOTION
Slow down time, down to the time-resolution equivalent of over 200,000+ frames per second, live, while generating orders of magnitude less data than traditional approaches. Understand the finest motion dynamics hiding in ultra fast and fleeting events.
Up to 200,000 fps (time resolution equivalent)
TOOLS
Metavision Studio is the perfect tool to start with, whether you own an EVK or not.
It features a Graphical User Interface allowing anyone to visualize and record data streamed by PROPHESEE-compatible Event-Based Vision systems.
It also enables you to read provided event datasets to deepen your understanding of Event-Based Vision.
Read, stream, tune events
Comprehensive Graphical User Interface
Main features
- Data visualization from Prophesee-compatible Event-Based Vision systems
- Control of data visualization (accumulation time, fps)
- Data recording
- Replay recorded data
- Export to AVI video
- Control, saving and loading of sensors settings
Metavision Studio is the perfect tool to start with, whether you own an EVK or not.
It features a Graphical User Interface allowing anyone to visualize and record data streamed by PROPHESEE-compatible Event-Based Vision systems.
It also enables you to read provided event datasets to deepen your understanding of Event-Based Vision.
Read, stream, tune events
Comprehensive Graphical User Interface
Main features
- Data visualization from Prophesee-compatible Event-Based Vision systems
- Control of data visualization (accumulation time, fps)
- Data recording
- Replay recorded data
- Export to AVI video
- Control, saving and loading of sensors settings
Metavision SDK is the largest set of Event-Based Vision algorithms accessible to date. High-performance algorithms are available via APIs, ready to go to production with Event-Based Vision applications.
The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.
Develop high-performance Event-Based Vision solutions
25 c++ code samples, 44 python classes 51 algorithms, 26 python samples, 16 tutorials and 11 ready-to-use applications
Main features
- Extensive documentation / code examples / training & learning material available
- Runs natively on Linux and Windows
- Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
- A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/********************************************************************************************************************** * Copyright (c) Prophesee S.A. * * * * Licensed under the Apache License, Version 2.0 (the "License"); * * you may not use this file except in compliance with the License. * * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed * * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * * See the License for the specific language governing permissions and limitations under the License. * **********************************************************************************************************************/ // This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event // counter and displayer by introducing some basic concepts of the Metavision SDK. #include <metavision/sdk/driver/camera.h> #include <metavision/sdk/base/events/event_cd.h> #include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h> #include <metavision/sdk/ui/utils/window.h> #include <metavision/sdk/ui/utils/event_loop.h> // this class will be used to analyze the events class EventAnalyzer { public: // class variables to store global information int global_counter = 0; // this will track how many events we processed Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed // this function will be associated to the camera callback // it is used to compute statistics on the received events void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) { std::cout << "----- New callback! -----" << std::endl; // time analysis // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and // the last event will have the highest timestamp Metavision::timestamp min_t = begin->t; // get the timestamp of the first event of this callback Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp // counting analysis int counter = 0; for (const Metavision::EventCD *ev = begin; ev != end; ++ev) { ++counter; // increasing local counter } global_counter += counter; // increase global counter // report std::cout << "There were " << counter << " events in this callback" << std::endl; std::cout << "There were " << global_counter << " total events up to now." << std::endl; std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds." << std::endl; std::cout << "----- End of the callback! -----" << std::endl; } }; // main loop int main(int argc, char *argv[]) { Metavision::Camera cam; // create the camera EventAnalyzer event_analyzer; // create the event analyzer if (argc >= 2) { // if we passed a file path, open it cam = Metavision::Camera::from_file(argv[1]); } else { // open the first available camera cam = Metavision::Camera::from_first_available(); } // to analyze the events, we add a callback that will be called periodically to give access to the latest events cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) { event_analyzer.analyze_events(ev_begin, ev_end); }); // to visualize the events, we will need to build frames and render them. // building frame will be done with a frame generator that will accumulate the events over time. // we need to provide it the camera resolution that we can retrieve from the camera instance int camera_width = cam.geometry().width(); int camera_height = cam.geometry().height(); // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps) const std::uint32_t acc = 20000; double fps = 50; // now we can create our frame generator using previous variables auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps); // we add the callback that will pass the events to the frame generator cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) { frame_gen.process_events(begin, end); }); // to render the frames, we create a window using the Window class of the UI module Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height, Metavision::BaseWindow::RenderMode::BGR); // we set a callback on the windows to close it when the Escape or Q key is pressed window.set_keyboard_callback( [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) { if (action == Metavision::UIAction::RELEASE && (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) { window.set_close_flag(); } }); // we set a callback on the frame generator so that it calls the window object to display the generated frames frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); }); // start the camera cam.start(); // keep running until the camera is off, the recording is finished or the escape key was pressed while (cam.is_running() && !window.should_close()) { // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core // and we push them into the window where the callback on the escape key will ask the windows to close static constexpr std::int64_t kSleepPeriodMs = 20; Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs); } // the recording is finished or the user wants to quit, stop the camera. cam.stop(); // print the global statistics const double length_in_seconds = event_analyzer.global_max_t / 1000000.0; std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl; std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl; if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short std::cout << "There were " << event_analyzer.global_counter / length_in_seconds << " events per second on average." << std::endl; } }
The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.
Develop high-performance Event-Based Vision solutions
32 c++ code samples, 112 python classes 63 algorithms, 47 python samples, 24 tutorials and 17 ready-to-use apps & tools
Main features
- Extensive documentation / code examples / training & learning material available
- Runs natively on Linux and Windows
- Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
- A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/********************************************************************************************************************** * Copyright (c) Prophesee S.A. * * * * Licensed under the Apache License, Version 2.0 (the "License"); * * you may not use this file except in compliance with the License. * * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed * * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * * See the License for the specific language governing permissions and limitations under the License. * **********************************************************************************************************************/ // This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event // counter and displayer by introducing some basic concepts of the Metavision SDK. #include <metavision/sdk/driver/camera.h> #include <metavision/sdk/base/events/event_cd.h> #include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h> #include <metavision/sdk/ui/utils/window.h> #include <metavision/sdk/ui/utils/event_loop.h> // this class will be used to analyze the events class EventAnalyzer { public: // class variables to store global information int global_counter = 0; // this will track how many events we processed Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed // this function will be associated to the camera callback // it is used to compute statistics on the received events void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) { std::cout << "----- New callback! -----" << std::endl; // time analysis // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and // the last event will have the highest timestamp Metavision::timestamp min_t = begin->t; // get the timestamp of the first event of this callback Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp // counting analysis int counter = 0; for (const Metavision::EventCD *ev = begin; ev != end; ++ev) { ++counter; // increasing local counter } global_counter += counter; // increase global counter // report std::cout << "There were " << counter << " events in this callback" << std::endl; std::cout << "There were " << global_counter << " total events up to now." << std::endl; std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds." << std::endl; std::cout << "----- End of the callback! -----" << std::endl; } }; // main loop int main(int argc, char *argv[]) { Metavision::Camera cam; // create the camera EventAnalyzer event_analyzer; // create the event analyzer if (argc >= 2) { // if we passed a file path, open it cam = Metavision::Camera::from_file(argv[1]); } else { // open the first available camera cam = Metavision::Camera::from_first_available(); } // to analyze the events, we add a callback that will be called periodically to give access to the latest events cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) { event_analyzer.analyze_events(ev_begin, ev_end); }); // to visualize the events, we will need to build frames and render them. // building frame will be done with a frame generator that will accumulate the events over time. // we need to provide it the camera resolution that we can retrieve from the camera instance int camera_width = cam.geometry().width(); int camera_height = cam.geometry().height(); // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps) const std::uint32_t acc = 20000; double fps = 50; // now we can create our frame generator using previous variables auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps); // we add the callback that will pass the events to the frame generator cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) { frame_gen.process_events(begin, end); }); // to render the frames, we create a window using the Window class of the UI module Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height, Metavision::BaseWindow::RenderMode::BGR); // we set a callback on the windows to close it when the Escape or Q key is pressed window.set_keyboard_callback( [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) { if (action == Metavision::UIAction::RELEASE && (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) { window.set_close_flag(); } }); // we set a callback on the frame generator so that it calls the window object to display the generated frames frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); }); // start the camera cam.start(); // keep running until the camera is off, the recording is finished or the escape key was pressed while (cam.is_running() && !window.should_close()) { // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core // and we push them into the window where the callback on the escape key will ask the windows to close static constexpr std::int64_t kSleepPeriodMs = 20; Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs); } // the recording is finished or the user wants to quit, stop the camera. cam.stop(); // print the global statistics const double length_in_seconds = event_analyzer.global_max_t / 1000000.0; std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl; std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl; if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short std::cout << "There were " << event_analyzer.global_counter / length_in_seconds << " events per second on average." << std::endl; } }
HARDWARE
BUY COMPATIBLE HARDWARE
Prophesee Evaluation Kits, Packaged sensor and Sony IMX636, realized in collaboration with Prophesee are fully compatible with Metavision Intelligence Suite as well as Century Arks’ SilkyEvCam – Powered by Prophesee.
BUILD COMPATIBLE HARDWARE
Through an extensive partnership program, Prophesee enables vision equipment manufacturers to build their own Event-Based Vision products.
Contact us to get access to source code, develop your own hardware or integrate your vision system with Metavision Intelligence.
FAQ
Do I need to buy an EVK to start ?
You don’t necessarily need an Evaluation Kit or Event-Based Vision equipment to start your discovery. You could start with Metavision Studio and interact with provided recordings first.
Which OS are supported ?
Metavision Intelligence suite supports Linux Ubuntu 18.04 / 20.04 and Windows 10 64-bit for PC X86-64 architecture). For other OS compatibilities, contact us or review our Open Source project, OpenEB.
What can I do with the provided license ?
The suite is provided under commercial license, enabling you download, use, build and even sell your own commercial application at no cost. Read the license agreement here.
Which Event-Based Vision hardware is supported ?
The software suite is compatible with Prophesee Metavision sensors and Evaluation Kits. It can also operate with compatible third party products.
Do you provide Python examples ?
Yes, we provide python sample code: https://docs.prophesee.ai/stable/samples.html
DOWNLOAD FREE TODAY
DOWNLOAD, BUILD, SELL - FOR FREE
Our software toolkit is available on Windows and Ubuntu PC. It is entirely free, without any time constraint. It includes a commercial-grade license, enabling you to design and sell your own Metavision Intelligence-based product, leveraging Prophesee Event-Based Metavision sensing technologies.