
Advanced MEMS microphones
Collect high-quality acoustic data for speech recognition, audio recording, and noise cancellation.
Advanced MEMS microphones
The Advanced MEMS microphones cater customer needs by gathering high quality, low noise acoustic data, which can be used for speech recognition, audio recording, and acquisition of surrounding noise for applications like noise cancellation.
SubSystem : Sensing and perception
Type of Component : Hardware / Software
Owner : IFAG
Expected TRL : 8
Licensing : Open source License TBD
Video

GRNEdge
Synchronize data streams for smart cities using a plug-and-play sensor.
GRNEdge
GRNEdge facilitates the synchronisation of different data streams (such as audio and video) for processing. It has the ambition to be a fully compliant plug-and-play sensor for smart city management This is possible once data-processing, such as audio-video anonymisation can also be done at the edge.
SubSystem : Sensing and perception
Type of Component : Hardware
Owner : GNR
Expected TRL : 6
Licensing : Proprietary

AVDrone
System that classifies crowd behavior while ensuring user privacy.
AVDrone
The proposed system will enable target users to classify the crowd behaviour without breaking any privacy issues.
SubSystem : Sensing and perception
Type of Component : Hardware / Software
Owner : UNS
Expected TRL : 5
Licensing : TBD
Video

AVRegistry
Store and access metadata of MARVEL Audio-Visual sources using a RESTful API compliant with Smart Data Models standard.
Advanced MEMS microphones
The AV Registry stores metadata of MARVEL Audio-Visual sources in compliance with the Smart Data Models standard and exposes this information through a RESTful API to all MARVEL components that need to consume AV data.
SubSystem : Sensing and perception
Type of Component : Hardware / Software
Owner : ITML
Expected TRL : 6
Licensing : TBD
Video

sensMiner
SensMiner Android app records and annotates environmental acoustics.
sensMiner
SensMiner is an Android app developed to record environmental acoustics as well as user annotations. While the audio is being recorded, the user can in parallel annotate it and store the corresponding segment in the phone memory.
SubSystem : Sensing and perception
Type of Component : Software
Owner : AUD
Expected TRL : 8
Licensing : Proprietary

EdgeSec-VPN
System that classifies crowd behavior with privacy.
EdgeSec-VPN
The proposed system will enable target users to classify the crowd behaviour without breaking any privacy issues.
SubSystem : Sensing and perception
Type of Component : Hardware / Software
Owner : UNS
Expected TRL : 5
Licensing : TBD
Video

EdgeSec-TEE
EdgeSec TEE ensures confidential computing for Python apps that process sensitive user data using Trusted Execution Environments.
EdgeSec-TEE
EdgeSec TEE offers confidential computing for python applications that process sensitive user data and is based on the technology of Trusted Execution Environments. It guarantees that the code itself and the data that needs to be processed are located inside protected and isolated execution environments that enable confidentiality and integrity, even if processed in untrusted environments.
SubSystem : Security, privacy and data protection
Type of Component : Software
Owner : FORTH
Expected TRL : 5
Licensing : Proprietary
Video

VideoAnony
VideoAnony component detects faces and number plates, and performs anonymization to simplify surveillance camera data analysis in public spaces.
VideoAnony
The VideoAnony component detects people’s faces and car number plates and performs anonymisation: by blurring in the initial phase and potentially by swapping faces at a later stage. The component should simplify the use of data recorded by surveillance cameras for monitoring and analysing public spaces.
SubSystem : Security, privacy and data protection
Type of Component : Software
Owner : FBK
Expected TRL : 6
Licensing : Open source License TBD
Video

AudioAnony
AudioAnony component anonymizes audio streams by replacing speaker voices with preserved speech features and environmental background.
AudioAnony
The goal of this component is to anonymize audio stream by replacing the speaker voice with another one while preserving the other speech fearure and the environmental background. Operating close to the microphones on edge device, or at the beginning of the procesing pipeline, audioanony limits possible privacy issues in the collection and processing of audio data. The component can deployed on a low-end edge device connected to the microphones to provide an anonymized stream in a transparent way
SubSystem : Security, privacy and data protection
Type of Component : Software
Owner : FBK
Expected TRL : 6
Licensing : Open source License TBD

Data Fusion Bus
The Data Fusion Bus (DFB) allows efficient and trustworthy transfer of heterogeneous data between connected components and permanent storage.
Data Fusion Bus
The Data Fusion Bus (DFB) is a fully customiseable system that supports a trustworthy and efficient transfer of streamed heterogeneous data between multiple connected components and a permanent storage. Within MARVEL, the DFB aggregates AI inference results from all E2F2C layers to post-process, store and re-distribute them to SmartViz and Data Corpus both in real time and as archived data.
SubSystem : Security, privacy and data protection
Type of Component : Software / Service
Owner : ITML
Expected TRL : 6
Licensing : TBD
Video

StreamHandler
StreamHandler is a powerful distributed streaming platform based on Apache Kafka, designed for big data applications. It provides high performance, interoperability, resilience, scalability, and security.
StreamHandler
StreamHandler is a high-performance distributed streaming platform for handling real-time data, based on Apache Kafka. The solution is geared toward big data applications and offers interoperability, resilience, scalability and security. In MARVEL, StreamHandler offers its audiovisual hadling capabilities, providing persistence for data streamed by microphones and cameras of the project. This way, the user can consult aural and visual evidence for events and anomalies detected by MARVEL’s AI components, and act accordingly.
SubSystem : Data management and distribution
Type of Component : Software / Service
Owner : INTRA
Expected TRL : 8
Licensing : Proprietary
Video

DatAna
DatAna is a versatile platform that allows for scalable acquisition, transformation, and communication of streaming data across different layers of computing.
DatAna
DatAna is a scalable data acquisition, transformation and communication platform based on the Apache NiFi ecosystem. DatAna allows processing of streaming data at different layers (edge, fog and cloud) and their routing towards other layers of the computing continuum. In MARVEL, data from the AI inference models is validated, transformed to be compatible with the Smart Data Models initiative and subsequently transmittted to the upper layers for further processing and storage.
SubSystem : Data management and distribution
Type of Component : Software
Owner : ATOS
Expected TRL : 6
Licensing : Open source License TBD
Video

Hierarchical Data Distribution (HDD)
HDD (High-speed Distributed Data) is a software component that optimizes the topic partitioning process in Apache Kafka, a popular distributed streaming platform.
Hierarchical Data Distribution (HDD)
HDD considers the problem of Apache Kafka data topic partitioning optimisation. Even though Apache Kafka provides some out-of-the-box optimisations, it does not strictly define how each topic shall be efficiently distributed into partitions. HDD models the Apache Kafka topic partitioning process for a given topic, and, given the set of brokers, constraints and application requirements, solves the problem by using innovative heuristics.
SubSystem : Data management and distribution
Type of Component : Methodology
Owner : CNR
Expected TRL : 6
Licensing : MIT
Video

CATFlow
CATFLOW is a powerful tool for anonymizing road camera data and presenting it in a user-friendly dashboard.
CATFlow
CATFLOW transforms road camera information to anonymous data. Through a few clicks, our dashboards then present this data and provide critical insights and detailed reports identifying a range of mobility types, trajectories, and junction turning ratios.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software
Owner : GNR
Expected TRL : 7
Licensing : Proprietary

devAIce
devAIce SDK is a powerful software development kit designed to help customers implement advanced audio analytics on their local premises.
devAIce
devAIce SDK is a full-blown audio analysis SDK meant to enable customers to perform intelligent audio analytics on their local premises.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software
Owner : AUD
Expected TRL : 9
Licensing : Proprietary

Visual Anomaly Detection
Visual Anomaly Detection is an advanced tool that utilizes cutting-edge deep learning models to detect abnormal events in video streams with high accuracy.
Visual Anomaly Detection
Visual Anomaly Detection implements efficient, state-of-the-art deep learning models to detect abnormal events in video streams. The unsupervised or weakly-supervised nature of this component decreases the effort required to gather and anotate the data, while maintaining high generalisation ability.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : AU
Expected TRL : 5
Licensing : Open source License TBD
Video
https://www.youtube.com/watch?v=–FFUl8esug

SED@Edge
SED@Edge is an innovative solution that enables the detection of urban acoustic events using deep learning models on low-cost, low-power microcontrollers with limited computational capabilities.
SED@Edge
SED@Edge implements state-of-the-art deep learning models for the detection of urban acoustic events on low-cost low-power microcontrollers with very limited computational capabilities. Operating on edge IoT devices located very close to the microphones, this solution brings a considerable energy and badnwith reduction with respect to cloud based solutions.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software
Owner : FBK
Expected TRL : 6
Licensing : Open source License TBD

Audio-Visual Anomaly Detection
Audio-Visual Anomaly Detection is a powerful component for detecting abnormal events in multimodal streams.
Audio-Visual Anomaly Detection
Audio-Visual Anomaly detection implements efficient, state-of-the-art deep learning models to detect abnormal events in multimodal streams. The multimodality of the AVAD places it on the cutting edge of research into the usage of multimodal data in the deep learning anomaly detection methods. The unsupervised or weakly-supervised nature of this component decreases the effort required to gather and annotate the data, while maintaining high generalisation ability.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : AU
Expected TRL : 5
Licensing : Open source License TBD

Visual Crowd Counting
The VCC uses computer vision and deep learning techniques to accurately count and classify vehicles passing through a given area.
Visual Crowd Counting
The VCC implements proven, state-of-the-art methods for crowd counting trained using transfer learning to allow high performance even on small datasets. The VCC component is accompanied by a host of auxilliary functionalities designed with ease of use and integration within the system framework in mind.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : AU
Expected TRL : 5
Licensing : Open source License TBD
Video

Audio-Visual Crowd Counting
The AVCC (Audio-Visual Crowd Counting) component is a solution for counting the number of people in a crowd from audio-visual data.
Audio-Visual Crowd Counting
The AVCC implements novel, state-of-the-art methods for crowd counting trained using transfer learning to allow high performance even on small datasets. The AVCC component is accompanied by a host of auxilliary functionalities designed with ease of use and integration within the system framework in mind.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : AU
Expected TRL : 5
Licensing : Open source License TBD
Video

Automated Audio Captioning
The MARVEL Dashboard provides an intuitive interface that allows users to visualize and interact with the data generated by audio captioning systems.
Automated Audio Captioning
AAC component implements a state-of-the-art method for automated audio captioning. Automated audio captioning is used in MARVEL to analyze continuous audio streams and to describe periodically the audio content with a textual description. These textual descriptions give a brief summary of actions in the scene.
SubSystem : Audio, visual and multimodal AI
Type of Component : Service
Owner : TAU
Expected TRL : 3
Licensing : Open source License TBD

Sound Event Detection
The VCC uses computer vision and deep learning techniques to accurately count and classify vehicles passing through a given area.
Sound Event Detection
SED component implements a state-of-the-art method for sound event detection. Sound event detection is used in the MARVEL to analyze continuous audio streams and to detect active sound events in the stream. Detected sound events give a detailed view of actions in the scene.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : AU
Expected TRL : 5
Licensing : Open source License TBD
Video

Sound Event Localisation and Detection
SELD component is a state-of-the-art method for sound event localization and detection.
Sound Event Localisation and Detection
SELD component implements a state-of-the-art method for sound event localization and detection. Sound event localization and detection is used in the MARVEL to analyze continuous audio streams, detect active sound events, and identify their location with respect to the audio capturing device. Detected and localized sound events give a detailed view of actions in the scene.
SubSystem : Audio, visual and multimodal AI
Type of Component : Software / Methodology
Owner : TAU
Expected TRL : 5
Licensing : Open source License TBD

GPURegex
FORTH platform includes a real-time stream processing engine designed to handle large volumes of data in real-time.
GPURegex
FORTH offers a real-time high-speed pattern matching engine that leverages the parallelism properties of GPGPUs to accelerate the process of string and/or regular expression matching. It is offered as a C API and allows developers to build applications that require pattern matching capabilities while simplifying the offloading and acceleration of the workload by exploiting the available GPU(s).
SubSystem : Optimised E2F2C Processing and Deployment
Type of Component : Software
Owner : FORTH
Expected TRL : 5
Licensing : Proprietary
Video

DynHP
DynHP, short for Dynamic Headroom Management, is a software solution that optimizes the use of computing resources in edge devices that have limited capabilities.
DynHP
DynHP makes it possible to execute the interference and training steps of audio-video real-time analytics on devices with limited compute capabilities at the edge of the network, which otherwise cannot be used due to insufficient memory or energy or processing resources.
SubSystem : Optimised E2F2C Processing and Deployment
Type of Component : Methodology
Owner : CNR
Expected TRL : 4
Licensing : MIT
Video

Federated learning
Federated learning is a distributed machine learning approach that allows multiple clients to collaboratively train a shared model while keeping their data private.
Federated learning
Federated learning offers an effective technique for model training with privacy protection and network bottleneck mitigation at the same time. As a result, such distributed approach has been widely accepted as an effective technique for addressing the problem of large, complex, and time- consuming training procedures, and is also particularly suited for edge computing.
SubSystem : Optimised E2F2C Processing and Deployment
Type of Component : Service
Owner : UNS
Expected TRL : 5
Licensing : Apache 2.0
Video

MARVdash
MARVdash is a platform that provides a web-based environment for data science in Kubernetes-based environments.
MARVdash
MARVdash is facilitating data science in Kubernetes-based environments, by integrating a web-based environment that includes an extensible set of productivity tools and services. It serves as a dashboard supplying the landing page for users, allowing MARVEL use cases and pilots services to be instantiated as orchestrated containers, and deployed via appropriate automation to execution sites selected.
SubSystem : Optimised E2F2C Processing and Deployment
Type of Component : Service
Owner : FORTH
Expected TRL : 5
Licensing : Apache 2.0
Video

MARVEL Data Corpus-as-a-Service
MARVEL corpus has the potential to create significant impact on both SMEs and the international scientific and research community. By providing access to these data assets, SMEs and startups can test and build innovative applications, potentially creating new business opportunities.
MARVEL Data Corpus-as-a-Service
MARVEL corpus will give the possibility for SMEs and start-ups to test and build on top of these data assets their innovative applications, thus creating new business by exploring extreme-scale multimodal analytics. In addition, by adopting an SLA enabled Big Data analytics framework, it is expected to maximise the impact that the MARVEL corpus will have on the international scientific and research community
SubSystem : System outputs
Type of Component : Service
Owner : STS
Expected TRL : 5
Licensing : Public
Video

Audio tagging
AT component in MARVEL implements a state-of-the-art method for audio tagging, which is used to analyze continuous audio streams and recognize active sound classes in the stream.
Audio tagging
AT component implements a state-of-the-art method for audio tagging. Audio tagging is used in the MARVEL to analyze continuous audio streams and to recognize active sound classes in the stream. Recognized active sound classes give a rough view of the scene.
SubSystem : Audio, visual and multimodal AI
Type of Component : Service
Owner : TAU
Expected TRL : 5
Licensing : Open source License TBD
Video