AI video analytics for Natural Language Search
Natural language search examines images and video using freely worded descriptions. It finds semantically relevant objects, activities and situations without requiring complex filter combinations.
Direct answer
What is natural language search?
Natural language video search translates a freely formulated description into a semantic search query and compares it with existing image or video footage.Instead of manually combining individual filters, users can search for a person with a red jacket, a white transporter in front of a loading zone or a person who can switch off an item.The system provides matching scenes as search candidates with camera, timing, and image context.
Natural Language Search
The search uses natural language to find visually matching scenes within images or video collections.
- ResultThe system reduces large video collections to semantically suitable sequences with camera, time and visual evidence.
- ResultsForensic search – Enterprise video analytics – Job and VLM processing
- PlatformsSynaEYE, SynaHUB, Vaidio
- QualityWe validate searches and results with real positive, negative and borderline cases under the actual camera conditions.
Scope of detection
Natural Language Search: capabilities and results.
Natural Language Search is not limited to deposited object classes. The search query can describe visible objects, properties, activities and contexts within a scene.
01 / 05
Visible objects and properties
Users can describe objects based on visible properties, such as a person with a red jacket, a black SUV, a person with a backpack or a yellow container on the floor. The system searches for scenes whose visible features fit as well as possible to the description.
02 / 05
Activities and Situations
Actions or visual situations can also be part of a search, such as a person making a bag, a vehicle in front of an entrance or several people in front of one door. This allows scenes to be researched for which no own event filter has been prepared.
03 / 05
Semantic similar scenes
The system does not only evaluate exact keywords. It looks for visual scenes that come as close as possible to the meaning of a description. A request therefore does not have to correspond exactly to the designation of an existing object class.
04 / 05
Time, camera and location
Time, camera, location or other metadata can supplement the free text search. For example, a query can be narrowed down to a white van at a certain loading zone between 6 pm and 10 pm.
05 / 05
Spontaneous visual issues
Operators can formulate new questions in natural language without first creating complex filter combinations or an own event type. The platform connects semantic image search with classic forensic filters.
From image to process
Natural Language Search in operational workflows.
The platform makes complex video archives accessible via natural language and reduces the manual research effort.
Formulate search
The user describes in natural language which object, which activity or situation is sought.
Compare material semantically
The platform compares the description with the available image or video footage and determines semantically matching scenes.
Limit search space
Camera, period, location or other analysis features can further reduce the search results.
Check hits in context
The system provides appropriate sequences with camera, time and image context. Authorised users check the hits directly in the original video footage.
Typical applications
Where is natural language search used?
Free-text search is particularly suitable for situations where the search is not yet determined before the start of the search, which classic filters or event categories are required.
Forensic search for security events
After a security event, operators can search relevant video inventories based on a description. Instead of checking each camera individually, you can search for certain clothing, visible vehicle features or a specific action.
Event-based follow-up search
A detected incident can be the starting point for further searches. Thus, similar situations before or after the event can be searched or checked whether a certain object was already visible earlier.
Ad-hoc questions
New questions do not necessarily exist as an analysis model or prepared event filter. Operators can spontaneously formulate visual questions and search existing video footage for suitable scenes.
Search for visible attributes
Visible features such as clothing, vehicle colour, vehicle type or other attributes can be described in natural language. For a clear determination of identity, however, separate and legally regulated functions are required.
Production, logistics and operation
Production, logistics or operating records can also be searched for visually described conditions, for example for pallets in the road or a missing container at a station.
Suitable platforms
Which platform supports natural language search?
The right platform depends on your task: evaluating ongoing camera images, checking individual states according to a schedule or centrally operating several analyses.
Operational video analytics
Continuous AI video analytics for live camera streams
SynaEYE continuously analyses existing cameras in real time and combines detection, event logic, alerts, tracking and forensics in a European edge AI platform.
The right choice if you:want to monitor security, safety or operational processes continuously with your cameras and process detected events immediately.
Periodic image checks
Using image questions as a recurring exam
With SynaHUB, a concrete image question becomes an inspection order. Camera, schedule or a connected event deliver the image; the platform evaluates it and documents the result.
The right choice if you:want to have a visual question repeated or answered on request.
Enterprise platform
Enterprise Visual AI with maximum functional breadth
Vaidio is an established enterprise platform with an extensive catalogue of standardised video analytics, powerful forensic search and a broad integration ecosystem.
The right choice if you:need a large, standardised enterprise environment with a broad analytics portfolio, multi-site operation and established VMS integrations.
SynaHUB performs image exams. For the search in a continuously indexed video archive, select a search solution designed for this purpose.
Compare platformsResponsibility and feasibility
Natural Language Search: limitations and requirements.
Natural Language Search works with semantic similarity. A hit therefore means that a scene matches the description – not that the search description has been proven beyond doubt. Image quality, formulation of the request, model and scene context influence the quality of results.
Search candidates instead of proof
Natural language search returns likely matching scenes. It does not guarantee complete results or that every suggested result is relevant to the use case.Formulation affects the result
Very general or ambiguous descriptions can produce a large number of low specific hits. More precise information about the object, activity, colour, location, or time period can improve the search.Visibility in the image
The system can only consider features that are sufficiently visible in the image. occlusion, low resolution, poor lighting or unfavorable perspectives can lead to not being found relevant scenes.Limits of language and vision models
Models can misinterpret visual content or semantically merge similar but actually different situations. Results therefore serve as research aid and must be checked in the original material.Storage and access rights
The search can only access image and video footage that is available within the defined retention periods and is released for the respective search.Assess data protection requirements for your specific use case. For EU locations, refer to the GDPR and the EDPB guidelines on video surveillance. Also consider the requirements of the EU AI Act, where applicable to your use case.
For private operators in Switzerland, the FDPIC guidance on video surveillance guidance.
Frequently asked questions
Natural Language Search explained in brief.
What is Free Text Search with AI video analytics?
Free-text search translates a freely formulated description into a semantic search request. The system compares the description with existing image or video footage.
What can be searched for in natural language?
For example, visible objects, clothing, vehicle characteristics, activities and situations can be described. The platform provides suitable scenes as search candidates.
How are search results provided?
First, the material is compared semantically. Subsequently, suitable sequences with camera, time and image context appear for human review.
Which platform does the free text search support?
SynaEYE, SynaHUB, Vaidio come into question depending on the task. With SynaHUB, a concrete image question becomes an inspection order. Camera, schedule or a connected event deliver the image; the platform evaluates it and documents the result. SynaHUB performs image exams. For the search in a continuously indexed video archive, select a search solution designed for this purpose.
What are the limits of natural language video search?
However, a semantically matching match does not automatically prove the search description. Therefore, relevant results are always checked in the original video context.
Does the free text search work with existing cameras?
In many cases yes. However, the decisive factors are perspective, resolution, light, compression and the visible size of the features sought.
Validate before scaling
Test natural language search with your cameras.
We assess the scene, image quality, data protection, integrations and response process using real data.
Discuss your use case
