Introduction: From Object Detection to Motion Understanding​

Detecting a vehicle in a video is not enough to determine whether it is speeding or approaching a hazardous area. To estimate its speed in km/h, the system must connect observations of the same vehicle over time and convert its movement in the image into a real-world distance.

The goal of this research was to maintain stable vehicle identities in low-FPS CCTV footage, calibrate the camera view against real-world road coordinates, and generate speed information that EVA can use.

Rather than simply replacing a tracking model, the research team implemented and validated the entire pipeline—from associating vehicles at low frame rates to converting pixel coordinates into real-world distances and presenting speed results in a verification interface. The demo below shows vehicle IDs, trajectories, a measurement area, and estimated speeds together.

1. Image Feature Extraction: Describing Vehicle Appearance​

In low-FPS footage, a vehicle may travel far enough between frames that its previous and current bounding boxes no longer overlap. Because position alone is insufficient to identify the same vehicle, each detected vehicle crop is converted into a feature vector that can be compared over time.

Why did we need three features?​

The visual cues that distinguish vehicles can be divided into shape, color, and contour. Each cue was reliable under different conditions, which made it difficult for a single feature to handle every CCTV environment consistently.

DINOv2: Representing Overall Vehicle Appearance​

DINOv2 extracts a deep feature that combines shape, texture, and visual components. Early in the research, we also evaluated Qwen visual features because they aligned naturally with EVA's VLM direction. VLMs are strong at semantic similarity, however, while tracking must distinguish individual vehicles that may look almost identical. When we compared the score distributions for matching and non-matching vehicles, DINOv2 separated the two groups more clearly, so we selected it as the default deep feature model.

HSV: Quantifying Color Distribution​

HSV separates hue, saturation, and value. After converting a vehicle crop from RGB to HSV, we accumulate H, S, and V values in predefined histogram bins to create a feature vector that represents which colors, and how much of each color, appear on the vehicle.

The HSV color space (left) and an example of accumulating pixel values by histogram bin (right)

We compare two vehicles' color distributions with histogram intersection, which sums the smaller value in each corresponding bin. Greater overlap produces a higher score, helping separate vehicles with similar types but different body colors.

HOG: Quantifying Contours and Intensity Boundaries​

HOG converts each vehicle crop to grayscale, then calculates the direction and magnitude of brightness changes at every pixel. It builds a histogram of gradient directions within small cells, normalizes neighboring cells together as blocks, and concatenates the results into a contour feature that relies less on color.

HOG represents the direction and strength of brightness changes as a directional histogram
Source: LearnOpenCV

No single feature was sufficient for every scene. HSV was especially effective when daytime color information was clear, while HOG contours provided a useful fallback when color cues weakened. We therefore normalized and fused the three scores using a baseline ratio of DINOv2 0.3 : HSV 0.5 : HOG 0.2.

This fusion allows the remaining features to support a match when one signal is disrupted by lighting or occlusion. Although the ideal weights can vary by scene, using all three provided a stable common baseline across different CCTV environments. The fused representation acts as a visual fingerprint for comparing past tracks with current detections.

2. Low-FPS Tracking: Linking IDs with Appearance and Motion​

Connecting vehicles based only on similar image features can swap their IDs when vehicles of the same color and type appear together. Motion alone also fails when a vehicle has just entered the scene or when the interval between frames is long. We therefore designed the tracker to use appearance as its primary matching signal and motion as supporting evidence.

Feature similarities are fused, and only candidates that pass motion and score gates are linked one-to-one

First, the tracker calculates fused-feature similarity for every combination of past track and current detection. For tracks with enough history, it adds a motion prior based on recent direction and speed to check whether the vehicle could plausibly have reached the observed position. Appearance finds candidates across large movements; motion checks whether each candidate is physically plausible.

Hungarian matching then assigns candidates globally without duplicates. An assignment is not accepted automatically. The tracker checks whether the similarity is high enough, whether it is clearly better than the next candidate, and whether the track and detection rank each other first.

If a pair fails those checks, the tracker removes that combination and searches the remaining candidates again. This recovers valid alternatives hidden by the first assignment without forcing an uncertain link. Only accepted observations update track memory, while new detections and tracks that have left the scene are managed separately to keep a single false detection from contaminating later matches.

Improving Tracking Performance at Night​

At night, both color and position cues become less reliable. Emergency lights and headlights can sharply change the HSV distribution of the same vehicle from one frame to the next. A detection box may also follow glare or a road reflection instead of the vehicle body, making the vehicle appear to move against its true direction. Because the next observation is already far away at low frame rates, neither appearance nor motion alone can reliably recover from this error.

Frames shown in chronological order. As the bounding box follows a glare blob, the observed ground-contact point moves opposite to the vehicle's direction of travel

Rather than relaxing every threshold, we adjusted the feature combination, candidate comparison, and track-retention policy for the causes of nighttime failure.

At 1 FPS on 12 UA-DETRAC sequences, these changes raised IDF1 from 64.0 to 75.3 and MOTA from 56.5 to 64.8, while reducing ID switches from 640 to 176. Replacing the deep feature accounted for about 15% of the improvement; the remainder came from the revised competition rules and nighttime-specific gates and lifecycle. The result reinforced that good features alone are not enough: the tracker also needs a consistent policy for when to reject and when to revisit uncertain associations.

3. Camera Calibration: Converting Pixels into Meters​

Tracking provides the pixel position of a vehicle and the time of each observation. Perspective, however, means that the same pixel displacement represents different real-world distances near and far from the camera. Pixel coordinates must therefore be projected onto a road plane measured in meters before speed can be calculated.

Perspective example: the same real-world distance of 5 m appears as different pixel lengths

The speed-estimation flow is straightforward.
Tracking first connects positions and capture times for the same vehicle, while calibration projects the vehicle's position from the image onto the road plane in meters. Finally, the system compares the real-world distance traveled with the elapsed time to estimate speed.

We initially investigated automatic calibration using monocular depth, an assumed average vehicle length, and vanishing points. Although convenient, this approach was sensitive to small estimation errors in CCTV scenes combining low frame rates, turning vehicles, and indistinct lane markings. It could also produce plausible but incorrect results whose cause was difficult for an operator to verify.

The final design therefore uses user-defined calibration. The user identifies reliable geometric references on the road plane, while the system calculates, validates, and stores the homography. It supports two input methods:

A homography unfolds the road area, which appears trapezoidal in the image, into a top-down plane. By matching image points with known road-plane coordinates, the system can convert vehicle positions from pixels into meters.

• Four corners and the real length and width of an area that is rectangular on the road plane
• At least four reference points with known real-world (X, Y) coordinates, including points matched to a site plan

BrnoCompSpeed is a public benchmark dataset for vehicle-speed estimation that provides fixed-road-camera footage with ground-truth vehicle speeds.
Using actual tracking trajectories from nine of its camera views, we measured the following mean absolute errors.

The resulting guideline is straightforward.
Place the calibration area where vehicles actually travel and make it as long as practical in the direction of travel. A one-meter input error creates a much larger relative error over a short area than over a long one. For projecting each vehicle onto the road plane, a single point at the bottom center of the bounding box was more stable than averaging multiple points.

4. Estimating Real-World Road Dimensions with a VLM​

User-defined calibration requires the real length and width of the selected road area. Direct measurement remains the safest source, but an operator may not know the appropriate region or initial values when configuring a new camera. We therefore evaluated whether a VLM could inspect the marked road area and surrounding scene, then suggest initial real-world dimensions.

In the experiment, the four corners were labeled P1–P4, and the measurement area was emphasized with a blue outline and semi-transparent red fill. The most stable results came from providing both the annotated image and the four pixel coordinates while keeping the question concise. An image alone or coordinates alone performed less consistently.

Example road-area overlay provided to the VLM

Using Qwen3.8-27B-FP8, we evaluated the improved input format on 21 images spanning three camera views across seven scenes.

Mean ratio error measures how far the VLM's estimated length and width deviate from the actual values on average; lower is better.

The revised input nearly halved the mean ratio error, but a 7.80-meter mean length error was still too large to use as the definitive metric scale for speed estimation. Longer perspective instructions and explicit reasoning did not improve every scene consistently.
At this stage, the VLM is therefore better suited as a configuration guide that a person can verify than as an automatic source of ground truth.

5. Speed Estimation Using Capture Timestamps​

The final speed estimator uses three pieces of information together:

• The track ID links observations of the same vehicle.
• The calibration result converts the bottom center of the bounding box into metric road coordinates.
• The actual capture time provides the elapsed time instead of relying on frame count.

The estimator maintains a short position history for each track, then checks how far apart the first and last points in the recent window are on the road plane. It calculates average speed by comparing that real-world distance with the actual capture-time difference.
Using capture timestamps preserves the correct time basis even when the input frame rate changes. An exponential moving average reduces small fluctuations caused by bounding-box jitter.

The current output is not an instantaneous speed. It is the average ground-plane speed between the endpoints of the recent window. Curved paths may therefore be underestimated, while camera shake and non-planar roads require additional handling.

Vehicle speed estimation demo

6. Productization Roadmap: From Research to an EVA Feature​

The purpose of this research was not limited to calculating vehicle speed. It was also intended to establish the foundation of an EVA feature that site operators can configure and use in practice. Combining vehicle speed and direction with EVA's detection zones, object information, and event rules enables the system to recognize scenarios such as the following.

This research was not limited to validating technical performance. It was conducted with productization in mind, considering the UX through which users configure the feature and verify its results. An operator can draw the vehicle measurement area directly on the camera view, receive initial length and width suggestions from the VLM, and confirm or correct them using site information or physical measurements. An overlay preview then displays the area, vehicle trajectories, and estimated speeds together so that the configuration can be checked immediately.

Once calibration is complete, users will be able to choose speed limits and target scenarios for each zone, then connect detected events to EVA alert channels and dashboards. A validated calibration can be reused for a fixed camera, while the interface can request recalibration after the camera position or zoom changes. The central UX goal is to hide unnecessary technical complexity while keeping the basis of the configuration and its results easy to verify.

The vehicle speed estimation technology developed through this research will go through productization—including site-specific setup guidance, VLM-assisted initial values, result-validation views, scenario configuration, and alert integration—and be released as an official EVA feature.

It is difficult to expect consistent decisions from a small VLM simply by asking it to “think deeply on its own.” A model may try to solve several problems within a single call, or revisit ambiguous evidence for too long without reaching a conclusion.

In this enhanced Thinking mode, we designed the reasoning flow itself as a set of explicit roles and structures instead of leaving every decision to the model’s internal reasoning. The pipeline establishes the criteria first, fixes the area to inspect, evaluates the scene from different perspectives, and leaves the final decision to code.

The goal is not to make the model think more. It is to make the order in which it should observe, judge, and question explicit.

1. Problems with the Previous Thinking Structure​

The previous development version judged several hazardous situations in the incidents block with a single VLM call.

The structure is simple, but assigning different problems to one call creates two issues.

1) Multiple problems are judged at once​

One call judges fall, smoke, and fire simultaneously. Even when some items are easy to determine, one ambiguous item can consume the entire reasoning budget.

2) The scope of reasoning is difficult to control​

Reasoning happens inside the model as a black-box process. In practice, the only value the application can adjust is the budget number; it is difficult to control which criteria the model revisits or what it repeatedly checks.

2. Limitations Revealed by Actual Thinking Traces​

Consider a situation in which smoke must be judged in a facility room filled with vapor. A model “thinking deeply” does not necessarily mean that it is finding new evidence.

Facility scene used for the smoke decision

Qwen3.5VL-9B-Thinking Langfuse Trace

After finding the white vapor, the model repeatedly reread the same rules to determine whether it was steam or a gas leak. However, no new visual evidence was added during the repetition.

The repeated thought, “Wait, let’s look closer,” produced zero new visual evidence.

The main problems identified in this trace are:

As a result, simply asking a small VLM to “think deeply on its own” is not enough. The system needs to guide what the model checks first, when it ends its judgment, and how it handles uncertainty.

3. Design Philosophy: Turn the Reasoning Flow into Structure​

The enhanced Thinking mode is built around three principles.

The new architecture first turns the scenario into explicit rules and then lets voters with different roles make independent judgments.

• The two voters run in parallel and independently; neither can see the other’s output.
• Every call uses an Instruct approach with enable_thinking: false.
• Each call receives only one decision problem.
• The final alert is triggered only when both voters are positive.

The LLM does not decide everything at once. Each stage focuses on its own role, while code controls the connections between stages and the final decision.

4. Core Mechanism: “I’m Looking Here Now”​

Before making a judgment, the model is required to output evidence of where it is looking. The role responsible for this is focus_locator.

focus_locator does not make the decision. It only suggests the object and location to observe. vote_context then uses both the full frame and a crop of the region specified by the Locator, while vote_skeptic independently verifies the scene from the full frame without relying on the Locator’s description.

Why does this improve performance?​

• An 8B-class VLM can easily judge a scene from its overall impression. Providing evidence from an explicit region first helps anchor the judgment to that evidence through grounding.
• A 20% margin crop enlarges the target while preserving surrounding context. This reduces mistakes caused by looking only at part of the target.
• Because the observed location is retained as text, an error can be analyzed as either an incorrect focus or an incorrect judgment.

5. Solving Failed Traces with Structure​

The new Pipeline roles and Gate address the problems found in the previous Thinking traces.

This changes the review process required for a decision into observable system stages instead of merely making the model’s internal reasoning longer.

6. Before / After​

The key change is not simply increasing the number of calls. It separates decision criteria, observation area, positive and counterevidence roles, and the final decision so that failure points can be traced.

7. More Flexible Scenario Expansion​

In the previous structure, the targets had to be fixed to smoke, fire, and falls because the LLM could easily move outside the intended decision scope.

In the new structure, even when a user edits the incidents block, a rule guide first organizes the decision criteria. Each voter judges only according to the generated criteria.

For example, users can enter requirements such as:

• “Smoke detected” → Detect black smoke
• “Fire detected” → Detect large flames occupying most of the region

The edited text is used to generate the Rule Guide, and each voter then uses those criteria. This reduces the need to modify the Router or decision code whenever a new scenario is added.

8. Performance Results​

Precision 0.8273: Suppressing False Positives​

In the SINGLE evaluation, the new structure substantially improved Precision.

Precision increased from 0.7316 to 0.8273, a gain of 0.0957 points. Recall decreased from 0.9006 to 0.7823. The AND Gate, which requires both voters to be positive before triggering an alert, suppresses false positives at the cost of conservatively blocking some true cases.

Category-level performance​

Smoke and flames reached 1.0000 across all metrics, while Fire Precision improved to 0.8085. In particular, false fire alerts were reduced to nine cases, reducing unnecessary alerts in operations.

In fall scenarios such as the example above, the system checks step by step whether a person is lying horizontally on the floor rather than simply crouching or sitting, and whether there is sufficient direct visual evidence.

Conversely, a red glow is not immediately treated as a real flame. The system reviews the lighting, reflections, flame shape, combustion clues, and whether smoke is present; without clear evidence, it returns alert: false.

Summary​

Instead of asking an 8B model to think more deeply, we laid out the path that its reasoning should follow — establish criteria, fix the focus, judge, and question.

As EVA's features and user requests grow, deciding which predefined route should handle each request becomes increasingly difficult. In EVA v3.1, we changed the chat architecture from a router-based agent to a tool-calling agent.

The previous architecture classified a request and sent it to a predefined processing route. The new architecture gives the LLM a registry of available capabilities, allowing it to select the tool required by the request and respond based on its execution result.

1. Why change the architecture?​

A router-based architecture has a clear advantage: each request type has an explicit processing path. However, as the number of features and question types increases, several limitations become visible:

• Adding a feature often requires changes to both the router and the graph branches connected to it.
• Manual content embedded in prompts and code becomes difficult to update and maintain.
• Questions about the current runtime configuration can become mixed with general feature explanations in the same Q&A flow.

2. Router-based agents and tool-calling agents​

The two architectures answer different questions.

In the router-based design, the code defines the available branches and selects one of them. In the tool-calling design, the LLM receives the available tool schemas and determines the tool required by the request.

The distinction is not simply about replacing a router with an LLM. It changes where the processing decision is made and how capabilities are added to the system:

Router-based agent
User request
  → Request-type classification
  → One predefined route
  → Node or Subgraph execution
  → Response

Tool-calling agent
User request
  → Intent analysis
  → Required tool selected
  → Tool execution
  → Execution result reviewed
  → Response

3. The Prepare–Decide–Tool–Compose–Finalize flow​

The new agent follows five conceptual stages:

User request
  ↓
Prepare
  ↓
Decide
  ↓
Tool
  ↓
Compose
  ↓
Finalize

For example, the request “Change the AI inference interval to 30 seconds” can be processed as follows:

Decide
  → Select set_detection_interval
  → Validate the input value: 30 seconds
  → Change the setting
  → Return the execution result
  → Generate the final response

Not every request needs every stage to perform a separate operation. The important point is that the agent has a consistent lifecycle in which tool selection, execution, and response generation are explicit.

4. Moving manual Q&A from prompts to Knowledge RAG​

The architectural change also affects how EVA answers questions about its manuals and features.

Prompt-based Chat​

In the previous design, guides were selected by request type and inserted into the prompt before generating an answer.

User question
  → Classify the question
  → Select a related guide
  → Include the guide in the prompt
  → Generate the LLM response

For example, a question about object-detection sensitivity might be classified as a terminology or app-feature question. The code would then select TERM_GUIDE or APP_GUIDE and include its content in the prompt.

This approach has several limitations:

• There is no separate document-retrieval step.
• The answer is limited to the guide content included in the prompt.
• Updating a guide may require changes to a prompt or code.
• The code must select the guide that matches the question type.

Tool-calling Chat​

In the new design, a manual question selects answer_eva_question. The tool embeds the question, searches the Knowledge store, and provides relevant pages as context for the answer model.

User question
  → Select answer_eva_question
  → Create a question embedding
  → Search related pages in Qdrant
  → Pass the retrieved page content as context
  → Generate the answer

For a question such as “How do I configure a detection scenario in EVA?”, the system searches for the manual page containing the relevant configuration procedure and generates an answer from that page.

This makes manuals an independently managed Knowledge source. Only the relevant pages can be retrieved, and metadata such as the file name and page number can be included in the result.

The trade-off is that retrieval quality now directly affects answer quality. The Knowledge pipeline and its evaluation become important parts of the chat system.

5. Page-level Knowledge Ingest​

RAG search requires manuals to be stored in a searchable form before users ask questions. The current ingestion flow is:

PDF or Markdown
  ↓
Extract text, tables, and figure information
  ↓
Build page-level content
  ↓
Create embeddings
  ↓
Store page vectors in Qdrant

At query time, the flow is:

User question
  → Create a query embedding
  → Search Qdrant by similarity
  → Return related pages
  → Use the pages as answer context

Why search by page?​

A page often contains information that is meaningful only when read together. For example, a single page may include:

• A feature description at the top.
• A configuration screen image in the middle.
• Important cautions at the bottom.

If the page is split into very small sentence-level chunks, these related pieces can be returned as separate search results. Page-level retrieval preserves the relationship between the description, configuration procedure, table, image explanation, and caution.

The goal is not simply to make chunks larger. It is to preserve information that is explained together within the same page.

The current structure finds related material at page granularity and passes the page text and metadata to the answer model as a ToolResult. This preserves the relationship between descriptions, configuration procedures, tables, and cautions while separating Knowledge retrieval from answer generation.

Extending Knowledge for customer environments​

Knowledge can include not only EVA manuals but also documents required for each customer environment. Customer operating manuals, equipment standards, work procedures, and safety and environmental regulations can be ingested so that answers are grounded in the standards and rules of the relevant site, rather than limited to general feature explanations.

For example, adding a safety and environmental regulations handbook to Knowledge makes it possible to ask not only whether a detection alarm occurred, but also which safety or environmental regulations apply, what the site response procedure is, and what follow-up reporting is required. Customer-specific documents can be managed as separate Knowledge sources while using the same tool-calling flow to search and answer according to each site’s operating rules.

6. End-to-end architecture​

The complete system has two related flows: an offline Knowledge ingestion flow and an online inference flow.

Offline Knowledge flow
PDF / Markdown
  → Extract text, tables, and figures
  → Create page-level text
  → Generate page embeddings
  → Store vectors in Qdrant

Online inference flow
User request
  → Prepare conversation history and current settings
  → Bind the available Tool Registry to the LLM
  → LLM creates the required tool call
  → Execute the selected tool
  → Return the ToolResult
  → Generate a natural-language answer when needed
  → Finalize the response

Tool-calling Chat​

The Tool Registry can contain different kinds of capabilities:

• Action Tools for changing application state.
• Q&A Tools for Knowledge-backed answers.
• Conversation Tools for general dialogue.
• Meta Tools for reading runtime information or other system state.

When answer_eva_question is selected, it starts the query-embedding and Qdrant-search flow. The retrieved page content is returned as a ToolResult and can then be used by the composing model.

This keeps the responsibilities separate: tools perform concrete operations, Knowledge retrieval provides evidence, and the language model generates the final response.

7. What the structural change enables​

The practical benefits of tool calling​

The value of tool calling is not simply that an LLM can call a function. Its main benefit is that request interpretation, concrete operations, Knowledge retrieval, and final answer generation have separate responsibilities and clear roles.

• Easier feature expansion: Add a tool and its schema to the registry instead of wiring every new capability into router branches and graph paths.
• Current information: Retrieve manuals and runtime state from Knowledge or state-lookup tools when needed instead of fixing them inside a prompt.
• Safer execution: Keep natural-language interpretation separate from state changes by applying validation, permissions, and execution conditions to each Action Tool.
• Better observability: Record and evaluate which tool was selected, which inputs were passed, and which search results were returned.

For the evaluated requests, the average response speed was about 30% faster than the previous router-based flow. The improvement came from reducing unnecessary classification steps and executing the required tool directly.

Easier feature expansion​

In the previous architecture, a new capability could require changes to router and graph branches. In the new architecture, the capability can be implemented as a tool and registered with its schema.

Before
  → Modify the router and graph branches

After
  → Implement a tool
  → Register its schema

Independent Knowledge management​

Manual content no longer needs to live in prompts or application code. It can be managed as Knowledge documents and reflected through the ingestion pipeline.

Clearer separation of request purposes​

The same user-facing topic can be connected to different backends depending on the request:

Current runtime value
  → Runtime setting lookup

Feature explanation
  → Knowledge search

Configuration change
  → Action Tool

Along with the response-speed improvement, the processing path becomes more flexible as the number of capabilities grows.

8. New operational considerations​

The architecture is more flexible, but it also introduces new areas that require monitoring and evaluation.

In particular, a successful tool-calling system needs to evaluate more than the final wording of the answer. It must also inspect whether the right tool was selected, whether its inputs were valid, whether the action was safe, and whether the answer was grounded in the right Knowledge page.

Closing​

The move from a router-based agent to a tool-calling agent is not simply a replacement of one chat component. It is a change from a system where code determines the request's processing path to a system where the LLM selects the capabilities and Knowledge required to handle the request.

With this structure, EVA can manage manuals as an independent Knowledge source and extend functionality by adding tools to a registry. At the same time, tool schemas, action safety, retrieval quality, document versioning, and end-to-end evaluation become essential parts of operating the agent reliably.

What matters is not simply whether every frame was processed. Operators also need to understand which processing stages are consuming resources, where resources are sitting idle, and whether processing is concentrated on particular cameras or Connections.

EVA Scope observes the end-to-end processing flow of the AI pipeline using Frame Journey data generated by EVA. It provides a comprehensive view of bottlenecks, latency, idle time, and processing imbalance, giving operators the evidence they need to determine how EVA can operate more reliably and efficiently within a limited operating environment.

This article introduces the architecture and key capabilities of EVA Scope, designed to observe frame-processing status in large-scale CCTV AI operations.

1. EVA Scope at a Glance​

View the status and Risk Score of numerous Connections on a single screen and prioritize the ones that need attention.

Connections List provides the following information for each Connection:

• Normal, caution, and warning status
• Latest Risk Score
• Bottleneck, Idle, Imbalance, and Error indexes
• Recent hourly status trends
• Total frame throughput
• Latest data collection status
• EVA App connection status
• Number of active Alerts

Operators do not need to inspect every Connection in sequence. They can start with the Connections that have the highest risk, check when the status began to change, and then move to the detailed analysis screen.

2. Signal-to-Insight Flow​

The core of EVA Scope is not simply displaying a large amount of data. It identifies signals that affect processing efficiency in a large volume of frame data and connects them to analysis at the Connection, camera, processing-stage, and individual-frame levels.

Connections List​

The first step is to review the list and status of all Connections, or EVA Apps. EVA Scope explains the factors that affect processing efficiency through four detailed indexes and sets Connection-level priorities with the Risk Score.

Operators can answer questions such as:

• Which Connection has the highest Risk Score?
• Is recent processing efficiency improving or deteriorating?
• Did throughput or latency change during a particular time period?
• Is data collection stable for a particular Connection?

Alert History​

When a key metric moves outside its configured threshold or the data collection status requires additional attention, the event is recorded in Alert History. Alert History displays the time period, Connection, status, and Risk Score associated with the problem.

Hourly Overview​

Select the Connection where the problem occurred. By setting a start and end time, operators can analyze the Image Frames that entered during that period. They can review the frames’ end states and status trends, processing times, and changes in key metrics to identify the problem. Historical periods can also be analyzed within the seven-day retention window.

Index Analysis​

EVA Scope provides four indexes that make up processing efficiency. Because it shows changes in each index together with its detailed components, operators can identify the factors behind a high metric rather than simply observing that the metric is high.

• Bottleneck Index: Processing delays and Queue waiting
• Idle Index: Idle levels in the Vision, Agent, and frame-ingestion stages
• Imbalance Index: Processing imbalance across cameras
• Error Index: Ratio of erroneous frames

This flow allows operators to go beyond checking an Alert and determine which stages and resources should be adjusted first to improve processing efficiency. EVA Scope does not decide the improvement plan on its own. Instead, it provides the evidence needed to consider follow-up actions such as reallocating resources, adjusting Queue policies, reviewing camera-specific processing policies, and improving the inference environment.

3. Why Observability Is Needed in Large-Scale CCTV Operations​

In a small-scale environment, checking a few cameras and a single EVA App can provide a reasonable understanding of the system status.

As the operation grows, however, the following problems emerge:

• The status of dozens of EVA Apps must be checked individually.
• Frame-processing status is distributed across hundreds or thousands of cameras.
• The final Drop result does not reveal where a failure occurred.
• It is difficult to connect the time of a problem to the original frame.
• Users must find the relevant data manually after noticing a problem.

An image-frame processing pipeline is not a simple request-response structure.

A single frame passes through multiple asynchronous stages. Some frames may be Dropped during processing or end in a different state. As a result, alerts visible in the UI alone are not enough to determine whether EVA is operating in an optimal state.

Frame Ingest
    ↓
Frame Queue
    ↓
Vision Inference
    ↓
Agent Queue
    ↓
Agent Inference
    ↓
Finished / Dropped / Error

To determine whether EVA is operating optimally, operators need to know which path each frame took and how much time it spent at each stage.

EVA Scope combines three types of information:

1. The final state of the frame
2. Timestamps for each processing stage
3. Processing context by Connection, camera, and time period

This makes it possible to answer questions beyond simply saying that inefficiency occurred:

• Which EVA App experienced the bottleneck?
• At what time did the load begin to increase?
• Is the bottleneck in Vision, Agent, or the Queue?
• Is the problem concentrated on a particular camera?
• Is the bottleneck still occurring?
• Did processing efficiency actually change after an action was taken?

4. EVA Scope Core Intelligence​

EVA Scope Core Intelligence is the core technology layer that collects and aggregates large volumes of frame data and transforms them into signals that can be used for operations.

4.1 Overall Architecture​

EVA Scope collects Image Frame Journey data from EVA Apps, stores it in a database, and analyzes it through a dashboard. In EVA Scope, a Journey refers to the processing path of an image frame from Ingest through Queue, Vision, and Agent to its final state.

Dozens of EVA Apps
        ↓
Hundreds to thousands of Cameras
        ↓
EVA App stats-dump (JSONL)
        ↓
FastAPI Backend
        ↓
Streaming Parser
        ↓
Hourly Rollup Accumulator
        ↓
ClickHouse
    ├─ Raw Frame Data
    ├─ Hourly Health Rollup
    ├─ Ingestion Ledger
    └─ Alerts
        ↓
Next.js Dashboard
    ├─ Connections List
    ├─ Alert History
    └─ Detailed Analysis

The current design considers dump files of approximately 50 MB per Connection per hour and a maximum of 100 Connections. At this scale, it is important to separate operational monitoring data from detailed analysis data instead of querying all Raw data every time.

4.2 Raw and Rollup: Data Collection and Storage Strategy at Scale​

When all Raw frame data is stored and queried continuously in a large-scale environment, storage requirements and query costs increase rapidly. EVA Scope processes data as a stream and separates Raw data from Rollup data during collection. With the rollup_only policy, the stream is consumed to create hourly aggregations, while Raw frames are not stored. With the full policy, the same stream is used for both Rollup aggregation and Raw batch inserts.

Streaming-Based Data Processing​

EVA App statistics are provided in JSONL format. EVA Scope reads the dump line by line and aggregates only the required information instead of storing the entire dump in the database or loading it into memory.

• Frame counts by status
• Frame counts and Drop counts by camera
• End-to-End latency
• Vision and Agent inference time
• Queue waiting ratio
• Actual time spent at each processing stage

This approach prevents memory usage from growing in proportion to the total size of the dump file.

Rollup Data​

Rollup is operational data aggregated by Connection and time bucket. It provides only the information needed for operations, processed from the data collected through streaming.

• Total frame count
• Finished, Filtered, Dropped, and Error counts
• End-to-End P50/P95 latency
• Frame and Drop counts by camera
• Bottleneck, Idle, and Imbalance indexes
• Risk Score

Connections List and Alert detection use Rollup data. This allows operators to quickly check the overall operational status without scanning thousands of cameras at the Raw-data level every time.

Raw Data​

Raw data is used for detailed analysis of individual frames. EVA Scope does not collect all Raw data; it collects and stores Raw data only for the Connection and time period that an operator wants to analyze.

• Inspecting error frames from a specific camera
• Tracing the processing path of an individual frame
• Checking Vision and Agent processing time
• Detailed analysis of a specific time period

EVA Scope also provides Connection-level storage policies.

Under the default policy, operations are centered on Rollup data. When an operator requests detailed analysis, Raw data for the required time period can be collected again. In other words, EVA Scope does not store all data in the same way. Operational data is aggregated lightly for fast queries, while data needed for root-cause analysis is separated so that it can be inspected in detail when needed. With this Rollup policy, EVA Scope maintains monitoring capabilities for all Connections while reducing storage requirements by 99%.

4.3 Key Metrics Design​

EVA Scope uses a range of metrics to diagnose operational efficiency and effectiveness in large-scale environments.

Closing​

The real challenge in a large-scale CCTV AI environment is not a lack of infrastructure but the opacity of the AI system’s internal behavior. If operators do not know where bottlenecks occur or which resources are sitting idle among the frames arriving from thousands of cameras, adding GPU servers will not solve the problem. Previously, it was difficult even to identify which EVA among the many running EVAs needed improvement. Even after finding one, analyzing system logs could take tens or hundreds of minutes. EVA Scope dramatically shortens this process by converting it into intuitive signals. By visualizing the entire Journey from image-frame ingestion to the final Agent inference, it enables operators to optimize the configuration and settings of the AI system based on clear data.

Beyond monitoring that simply checks whether “the server is alive,” and beyond an operations system that checks “where a failure occurred,” EVA Scope enables operators to determine what to observe and what to improve so that EVA can operate stably and efficiently within limited system resources.

This is the value that EVA Scope aims to provide in large-scale CCTV AI operations.

As the number of cameras and sites grows, the volume of alerts grows with it. When false positives accumulate, field operators stop trusting the alerts. Checking only whether cameras are connected or whether the system is running is not enough to improve detection quality; operators also need to know which cameras and scenarios need attention and whether a configuration change actually worked.

EVAmentor is an operational solution that measures performance from EVA decisions and user feedback, diagnoses problems, finds directions for improvement, and validates the results. It goes beyond monitoring system status by providing mentoring that helps operators understand the causes of false positives and decide which feedback and settings to apply. EVAmentor’s role is to help continuously refine detection quality.

1. EVAmentor at a Glance​

EVAmentor connects the operational cycle of finding problems → diagnosing causes → improving → validating in one flow. Operators can review alerts, feedback, and configuration-change history using the same criteria and decide what to check and improve first.

Overview — The First Screen for Understanding Detection Status​

For each site, operators can see the number of cameras, alert load, alert accuracy, feedback effectiveness, and recent alerts at a glance. The screen shows not only the number of alerts, but also the share judged to be real risks and how much user feedback contributes to reducing false positives. This makes it easier to understand the current detection quality and identify what needs attention first.

Action Required — Deciding What to Review First​

Instead of manually checking dozens or hundreds of cameras and scenarios, EVAmentor automatically classifies their status based on recent data and review history. Operators can start with items that require immediate action, such as disconnected data or concentrated false positives, instead of scanning the entire list from the beginning.

Accuracy Analysis ① — Understanding Performance Trends and Causes​

Accuracy Analysis shows alert accuracy and feedback effectiveness together with their calculation logic, and lets operators drill down into detailed metrics by scenario and camera. Because the source alerts and review results behind each metric can be examined, the data provides a basis for decisions such as keeping or removing a scenario.

Change Impact Analysis places scenario edits and model-replacement events on top of the accuracy trend. Operators can therefore understand the relationship between performance changes and configuration changes through recorded history rather than memory or guesswork.

From this screen, operators can confirm:

• when accuracy changed,
• what was changed at that point, and
• whether the change actually had an effect.

Accuracy Analysis ② — Comparing Detection Settings with Data​

The objects and thresholds used before VLM judgment also affect the volume and quality of alerts. For example, detecting missing helmets through a person object or a bare head object can produce different results for the same scenario. By comparing the accuracy and alert counts of baseline and alternative settings side by side, operators can use data to decide which camera settings to change first.

Feedback Analysis — Checking the Feedback Loop​

EVA suppresses similar false-positive alerts based on user feedback. EVAmentor helps verify whether this filter is working correctly:

• Did it correctly filter actual false positives?
• Did it incorrectly filter real risks?
• Which reference images contributed most to the decision?

Each event retains its reference image and similarity score, making the reason for filtering traceable. Operators can quickly find incorrectly filtered risks, refine feedback, and check whether a particular reference image is causing excessive filtering.

A/B Testing — Improving Without Waiting for False Positives to Recur​

After changing a scenario, operators do not have to wait for the same situation to occur again in the field. They can use previously collected false-positive images to validate a proposed change immediately, even when the false positive occurs only rarely.

1. Re-evaluate past false-positive images with the existing scenario.
2. Improve the scenario using an AI-generated draft or direct editing.
3. Apply the revised scenario to the same images and compare the results.

Operators can confirm before deployment whether the change reduces false positives while preserving true positives. This moves the process from discovering side effects after deployment to checking them before the change reaches the field.

Image Gallery — Turning Review into Real Improvement​

The Image Gallery brings alerts and filtered events together for true-positive and false-positive labeling. Unreviewed items can be compared visually and processed individually or in batches using conditions. Labels do not remain only in EVAmentor; they are also reflected in the corresponding EVA App analysis results and become evidence for future detections.

2. The Value EVAmentor Creates​

Managing the Performance of Hundreds of Cameras​

Instead of opening every alert one by one, EVAmentor automatically aggregates performance by site, camera, and scenario and presents an action priority. Even as the number of cameras grows, operators can use the same criteria to review the most important problems first. This changes the daily operation from opening alerts at random to working through the highest-priority items in the action list.

Making Invisible Problems Visible​

EVAmentor surfaces incorrect feedback filters, review gaps, disconnected cameras, and scenarios that no longer fit the site through clear statuses and metrics. It also exposes problems that are easy to miss, such as a blind spot where alerts keep arriving but nobody reviews them, or a filter that suppresses real risks along with false positives. Because the basis for each metric is shown, operational decisions can be explained and defended.

Reducing Review Burden and Building Operational Assets​

Operators can label images immediately and process selected items in batches. Since the system shows what should be reviewed first, more important events can be handled within the same amount of time. The resulting review history becomes an operational asset for comparing performance before and after a scenario change and tracking long-term trends.

Making Improvement About Validation, Not Guesswork​

Past false-positive images, object and threshold comparisons, AI-generated scenario drafts, and change-impact analysis allow teams to measure the effect from before a change is made through after it is applied. The process shifts from changing settings based on experience and waiting for field alerts to making data-driven changes and validating them immediately.

Human Decisions Make the System Smarter Again​

Labels and review results from EVAmentor are reflected in the next decisions made by EVA App. As their effect is measured again, a positive cycle of measurement → diagnosis → improvement → application → remeasurement is created. With each iteration, decisions that do not fit the site decrease and the foundation for trusting field alerts grows stronger.

3. Closing​

If EVA is the system that watches over the field, EVAmentor is the solution that watches over EVA and makes it more accurate.

Detection results · Feedback collection
    ↓
Performance metric aggregation
    ↓
Automatic classification of items requiring action
    ↓
Evidence image review · True/false-positive confirmation
    ↓
A/B testing · Detection-setting comparison
    ↓
Re-measurement of improvement effects

Detection quality is not created by a single tuning exercise. It is built by operating this loop continuously.

EVA does not rely solely on model optimization or application-level scheduling. It determines the optimal configuration for each deployment environment by considering the number of GPUs installed in the server, GPU memory capacity, MIG partitioning availability, MPS effectiveness, and the placement of Vision Workers and vLLM instances.

In other words, EVA is not just a service that runs AI models. It maximizes system resource efficiency by considering hardware-level GPU configuration and the behavior of the Serving Framework. This allows EVA to reliably process requests from many cameras, even with limited server resources.

In this article, we compare the effects of MIG and MPS based on actual EVA experiment data from the following three perspectives.

• MIG effectiveness on a multi-GPU server: PRO 5000 x3
• MIG effectiveness on a single-GPU server: PRO 6000 x1
• MPS effectiveness in an environment with many Vision Workers

Through this analysis, we aim to provide practical criteria for determining which MIG/MPS configuration is most suitable for EVA operation, rather than applying MIG or MPS unconditionally.

• MIG(Multi-Instance GPU): A feature that partitions a single physical GPU into multiple independent GPU instances. It can reduce resource contention by placing Vision and vLLM workloads on separate instances.
• MPS(Multi-Process Service): A feature that allows multiple CUDA process requests to be coordinated through a single server process. It can reduce context-switching overhead in multi-process environments.

1. EVA Inference Architecture​

An EVA server runs two major inference layers at the same time.

• Vision: Processes various object detection models such as RT-DETRV2, Owl-v2, OmDet, and LLMDet in parallel through multiple Worker processes.
• vLLM(VLM Serving): Receives Agent requests, decomposes user-defined scenarios into multiple tasks, and determines whether the scenario condition is met through multi-step inference.

The key point in EVA’s inference architecture is that Vision and vLLM use the GPU in different ways.

For models used by many cameras, EVA assigns more Workers to those models so that inference requests can be processed in parallel by model type. On the other hand, vLLM processes multiple requests together through continuous batching, so simply increasing the number of instances does not always lead to higher throughput.

Therefore, EVA determines whether to apply MIG and MPS based on the server configuration and workload characteristics.

2. Experiment Environment​

2.1 Server Configuration​

The experiment was conducted using the following two GPU server configurations.

2.2 Service Placement​

Because Vision and vLLM have different GPU usage patterns, it is important to determine which GPU or MIG instance each workload should be placed on.

In general, Vision is configured as the area responsible for object detection requests, while vLLM is configured as the area responsible for Agent VLM-based reasoning requests. When MIG is applied, each remaining GPU resource or MIG slice, excluding the GPU or MIG slice used by Vision, is assigned one vLLM instance.

For example, when MIG is applied on the PRO 5000 x3 server, six 24GB MIG slices are created. One slice is assigned to Vision, and one vLLM instance is placed on each of the remaining five slices. The same approach is used on the PRO 6000 x1 server: among four 24GB MIG slices, one is assigned to Vision and the remaining three are assigned to vLLM instances.

This configuration was designed to evaluate whether MIG can improve actual throughput when Vision and vLLM are separated as much as possible and vLLM instances are evenly placed on the remaining GPU resources.

3. Metric Definitions​

In this article, Vision and Agent throughput are compared using the following metrics.

The conversion formulas are as follows.

• Vision req/s = Total Vision requests processed in 1 hour / 3600
• Agent req/min = Total VLM responses in 1 hour / 60

4. MIG Effect on PRO 5000 x3​

4.1 Measurement Results​

In the PRO 5000 x3 environment, we measured the performance impact of the MIG-enabled configuration under the following conditions.

• Test scenario: Single scenario
• Vision model configuration: Single Vision model

Therefore, the results below should not be directly compared with the PRO 6000 x1 environment. Instead, they should be interpreted as the performance change within the same PRO 5000 x3 environment before and after applying the MIG-enabled configuration.

4.2 Interpretation​

In the multi-GPU PRO 5000 x3 configuration, increasing the number of vLLM instances through MIG did not produce a meaningful improvement in vLLM throughput.

Because vLLM was already efficiently handling concurrent requests through continuous batching, increasing the number of instances did not directly translate into higher throughput. In addition, the reduced available resources per instance caused by MIG partitioning and the overall workload placement changes appear to have contributed to the decrease in Vision throughput.

From an operational perspective, the following factors should also be considered when applying MIG.

• MIG partitioning policy management
• Instance-level monitoring
• Reassignment and recovery procedures in case of failure
• Workload-specific instance size adjustment

Therefore, in a multi-GPU environment such as PRO 5000 x3, it is more appropriate to start without MIG by default and consider MIG only when resource contention between Vision and vLLM is clearly identified.

5. MIG Effect on PRO 6000 x1​

5.1 Measurement Results​

In the PRO 6000 x1 environment, we measured the performance impact of the MIG-enabled configuration under the following conditions.

• Test scenario: Two or more complex scenarios
• Vision model configuration: Multiple Vision models

Therefore, the results below should not be directly compared with the PRO 5000 x3 environment. Instead, they should be interpreted as the performance change within the same PRO 6000 x1 environment before and after applying the MIG-enabled configuration.

5.2 Interpretation​

In the single-GPU PRO 6000 x1 configuration, the effect of MIG was clearly observed.

Without MIG, Vision Workers and vLLM share a single physical GPU. In this case, many Vision Workers repeatedly generate short inference requests, while vLLM processes relatively large inference workloads. As a result, GPU ownership can switch frequently between workloads.

By applying MIG, the GPU resources used by Vision and vLLM are isolated into hardware-level independent instances. This reduces resource contention between workloads and allows each inference pipeline to run more stably.

These results show that MIG can be an effective option in high-density environments where Vision and vLLM must run together on a single GPU. In particular, the more different the GPU usage patterns of Vision and vLLM are, the greater the benefit of hardware-level isolation through MIG can be.

6. MPS Effect in an Environment with Many Vision Workers​

Vision models run through multiple Worker processes that send requests to the GPU at the same time. When the number of Workers increases, GPU ownership can switch frequently between processes.

By applying MPS, multiple CUDA process requests can be coordinated through a single MPS server. This can reduce context-switching overhead and improve GPU utilization in multi-process environments.

In this experiment, we compared the effect of MPS on the PRO 5000 environment without applying MIG.

With MPS enabled, total Vision throughput increased by approximately 9.3%, from 23.770 req/s to 25.980 req/s.

Among the models, Owl-v2 showed the largest throughput improvement, while RT-DETRV2, OmDet, and LLMDet also showed slight improvements. This indicates that MPS can help coordinate multi-process requests more stably in environments with many Vision Workers.

7. Final Conclusion​

This experiment confirmed that MIG and MPS are not features that should be applied uniformly in every environment. Instead, they should be applied selectively depending on the GPU configuration and workload characteristics.

7.1 Multi-GPU Server: PRO 5000 x3​

In an environment such as PRO 5000 x3, where multiple physical GPUs are available, Vision and vLLM can be separated at the physical GPU level. In this case, additionally applying MIG to increase the number of vLLM instances showed limited throughput improvement.

• vLLM already handles concurrent requests efficiently through continuous batching
• Increasing the number of instances does not directly lead to higher throughput
• MIG partitioning can increase operational complexity and resource fragmentation
• The default recommendation is to start without MIG

7.2 Single-GPU Server: PRO 6000 x1​

In an environment such as PRO 6000 x1, where Vision and vLLM must run together on a single physical GPU, MIG showed a significant effect.

• Vision and vLLM have different GPU usage patterns
• Sharing a single GPU can increase context-switching overhead
• MIG reduces resource contention between workloads
• MIG should be considered first for high-density single-GPU configurations

7.3 Environment with Many Vision Workers​

MPS can be effective in environments with many Vision Workers.

• Many Workers generate GPU requests concurrently
• GPU ownership switching between processes can increase overhead
• MPS increased total Vision throughput by approximately 9.3%
• MPS should be considered as a default option for Vision-heavy servers

8. Operational Recommendations​

In summary, EVA does not treat MIG and MPS as simple on/off features. It selects the optimal configuration for each environment by considering server architecture, Vision/VLM placement, Worker count, vLLM execution behavior, and operational complexity.

9. References​

The following materials were referenced when organizing the analysis framework for this article.

• NVIDIA Technical Blog: Getting the Most Out of the NVIDIA A100 GPU with Multi-Instance GPU https://developer.nvidia.com/blog/getting-the-most-out-of-the-a100-gpu-with-multi-instance-gpu/
• NVIDIA Technical Blog: Boost GPU Memory Performance with No Code Changes Using NVIDIA CUDA MPS https://developer.nvidia.com/blog/boost-gpu-memory-performance-with-no-code-changes-using-nvidia-cuda-mps/
• NVIDIA Documentation: Multi-Process Service (MPS) https://docs.nvidia.com/deploy/mps/latest/index.html
• vLLM Official Blog https://vllm.ai/blog
• Anyscale Technical Blog: vLLM Throughput Analysis Based on Continuous Batching https://www.anyscale.com/blog/continuous-batching-llm-inference

In large-scale camera environments, simply using a higher-performance GPU is not enough. The system must maintain stable streaming for all cameras while processing AI inference requests from multiple cameras within limited CPU, memory, network, and GPU resources.

If the AI pipeline becomes biased toward a specific camera or model, some cameras may not be analyzed properly, or the delay between event occurrence and alarm delivery may increase. For this reason, EVA optimizes its infrastructure across the entire pipeline, from video ingestion and AI inference to streaming delivery.

In this article, we introduce the key infrastructure optimization technologies EVA applies to reliably support large-scale camera environments.

1. VM·VLM Separated Architecture​

EVA does not analyze every video frame with a high-cost VLM. Instead, a VM(Vision Model) first checks whether the target object exists and whether basic conditions are met. Only when further reasoning is required does EVA run the VLM(Vision Language Model).

For example, in a PPE non-compliance detection scenario, frames without a person or frames unrelated to the target condition are filtered out at an early stage. VLM inference is performed only when a person is detected and the system needs to determine whether protective equipment is being worn.

In the initial architecture, GPU computation increased rapidly because every frame was processed mainly through the VLM. In the current architecture, the VM first filters the targets that require further analysis, significantly reducing the number of VLM calls.

With this structure, EVA improved the number of cameras that can be supported on the same server from around 20 to up to 100. This figure is based on internal validation comparing the initial VLM-centric structure with the current architecture, where VM filtering is applied before invoking the VLM only when necessary.

2. Task Decomposition and GPU Parallel Processing​

EVA does not process inference requests from multiple cameras as one large task. Instead, it decomposes the scenario reasoning process into smaller tasks such as detection step evaluation, exception checking, image description generation, and vectorization.

These decomposed tasks are distributed and processed in parallel across multiple inference instances. This design prevents any single task from blocking the entire pipeline when requests occur simultaneously from many cameras.

The key idea behind this approach is to terminate unnecessary inference early. If an early step determines that the alarm condition is not met, EVA does not proceed with additional VLM calls, image description generation, or vectorization.

As a result, GPU resources can be focused only on tasks that require actual reasoning. In real operating environments, this improved scenario throughput from 360 to 1,192 scenarios per hour, more than a 3x increase.

3. Inference Optimization Based on Detection Modes​

Processing every scenario in the same way increases unnecessary GPU usage. EVA analyzes the characteristics of each user-defined scenario and automatically selects the most appropriate detection method.

Scenarios that only require simple object presence checks are handled mainly by the VM, while scenarios that require complex contextual reasoning are processed using VLM-based inference.

For example, a scenario such as “Notify me when a person is visible” does not require a high-cost VLM and can be processed with the VM alone. On the other hand, a scenario such as “Notify me when a worker enters a dangerous area without wearing a safety helmet” requires additional reasoning because it must evaluate object presence, PPE status, and spatial context together.

By using only the level of model required for each scenario, EVA reduces GPU usage while maintaining detection performance.

4. Dynamic Worker Allocation by Model​

EVA dynamically allocates Workers based on the models in use and the number of cameras assigned to each model.

For example, if many cameras are using OMDet at a certain point in time, EVA assigns more Workers to that model. Models with lower usage are kept with minimal resources. This prevents GPU resources from being overly concentrated on a specific model and helps maintain stable overall GPU utilization.

If Workers are allocated statically, a model may occupy resources regardless of actual usage, or a heavily used model may not have enough processing capacity.

EVA adjusts the number of Workers based on request volume by model, keeping the rate of AI inference target frames missed due to processing delay within 10%. This figure is an operating metric managed under an internal load test environment with 100 cameras and simultaneous AI inference requests.

5. Priority Queue-Based Request Scheduling​

In large-scale camera environments, certain cameras may temporarily generate a large number of requests. If requests are processed in a simple FIFO order, one camera may occupy excessive system resources, delaying inference for other cameras.

EVA manages inference requests by camera using a priority queue so that all cameras can access the AI pipeline fairly.

This structure prevents inference for other cameras from being delayed even when requests temporarily spike from a specific camera. It also prevents outdated frames from being processed too late and causing incorrect alarms, helping maintain the stability of the real-time AI pipeline.

6. Dynamic FPS Control Based on Operating State​

EVA dynamically adjusts video collection and processing frequency based on each camera’s operating state.

A simple connection state, an AI inference state, and a live streaming state all require different levels of processing. If all cameras are processed at the same FPS, unnecessary CPU, GPU, and network usage increases.

For example, in a simple connection state, EVA collects only the minimum number of frames needed to maintain the connection. When AI inference is required, only the frames needed for analysis are selected and processed. Higher-FPS streaming is performed only when a user is watching the live video.

In particular, EVA does not decode every collected frame or use every frame for AI analysis. It selectively extracts only the frames needed for inference, minimizing unnecessary video collection, decoding, preprocessing, and inference operations.

Internal validation showed that, under the same server specifications, CPU usage for 50 cameras in Monitoring On state was reduced from around 100% to around 65%, resulting in an approximately 35% reduction in CPU resource usage.

7. Multi-User Streaming Optimization​

In monitoring environments, multiple users often watch the same camera stream at the same time.

In a conventional structure, as the number of users increases, decoding, preprocessing, and encoding may be repeatedly performed for the same video stream, significantly increasing CPU and memory usage. EVA reduces this overhead by sharing video processing results for the same camera.

EVA performs decoding, preprocessing, and encoding only once for the same camera, then shares the generated stream with multiple users. As a result, even when the number of users increases, the additional cost is mainly limited to network transmission.

This allows EVA to maintain stable streaming quality while minimizing server resource usage, even when many users monitor video streams simultaneously.

Closing​

EVA’s infrastructure efficiency does not depend solely on single-GPU performance. It is the result of optimizing the entire pipeline, including video ingestion, frame selection, AI inference, request scheduling, and streaming delivery.

EVA combines the following technologies to reliably support more cameras on the same server environment.

Ultimately, EVA’s large-scale camera capacity is not achieved by a single optimization technique. It is the result of designing the AI model structure and system infrastructure together to use limited server resources more efficiently.

EVA will continue to enhance its infrastructure optimization technologies to reliably support more cameras, more complex scenarios, and more diverse operating environments.

EVA has operated Vision Model, Vision Language Model, and Agent pipelines in GPU-based environments. To secure higher power efficiency and more cost-efficient scalability, we validated and optimized whether EVA could also run at commercial-service quality in a Rebellions NPU environment.

Running EVA on NPU was not just a model-porting task. To operate a GPU-centric AI service pipeline stably on NPU, we had to optimize model compilation, input resolution, parallel processing, and CPU-NPU resource placement together.

Key Optimization Points Validated for NPU Productionization​

1. Optimizing Models for NPU Execution Structure​

The first question in NPU environments was whether each model could be transformed into a structure suitable for NPU execution. In GPU environments, PyTorch-based models can be run relatively flexibly. In NPU environments, however, model structure and input formats must be explicitly defined through a dedicated compiler.

Because EVA's Vision models handle diverse camera inputs and scenarios, we had to optimize not only model execution itself, but also preprocessing, postprocessing, input shape policy, and coordinate restoration.

To address this, EVA reorganized model structures into NPU-friendly forms, fixed input shapes, and rebuilt image preprocessing and output-to-original-coordinate restoration in line with the EVA pipeline.

This was not merely about placing a model on NPU. It was an optimization effort to provide stable response times and consistent inference outputs in real production environments.

2. Balancing Input Resolution and Fine-Grained Decision Quality​

Raising inference speed on NPU requires input-resolution optimization. But if full images are uniformly downscaled across all scenarios, performance can degrade in cases that require fine-grained decisions.

This trade-off is especially important in PPE compliance scenarios that require precise recognition of small regions. Helmets, masks, goggles, and gloves occupy small portions of an image, so downscaling the full frame can weaken critical visual details.

EVA did not solve this by simply raising full-image resolution. Instead, EVA first identifies the region required for decision-making, then crops that region and reconstructs it at the model's required input resolution.

For example, when judging PPE compliance, EVA does not pass the full frame directly to VLM. It crops around the person region and scales that crop to the model input size before inference.

This approach offers several advantages.

• It reduces inference cost by avoiding high-resolution processing of the full image.
• It preserves sufficient resolution in the region the model actually needs to inspect.
• It mitigates accuracy loss in small-object or fine-state judgment.

In other words, EVA does not simply reduce resolution in NPU environments. It distinguishes between regions the model must inspect and regions it does not.

3. Detection Quality Validation with Real Operational Data​

For NPU productionization, speed alone is not enough. Detection quality must be validated as well. Even if a model runs stably on NPU, deployment requires confirming that decision quality is maintained relative to GPU environments.

EVA built its own validation dataset from real detection data collected across operational environments. Rather than relying on public benchmarks, we validated performance against the scenarios EVA actually needs to judge.

The validation set included field scenarios such as the following.

Using this data, we validated identical scenarios in both GPU and NPU environments. As a result, we confirmed that NPU could maintain quality at a level similar to GPU across major scenarios.

That said, scenarios requiring fine visual detail for small objects remain sensitive to resolution changes, so combining this with the crop-based input optimization described above is important.

4. Task Decomposition to Improve VLM Call Efficiency​

In NPU environments, it is often more effective to filter requests that truly require high-cost inference than to process everything as one large inference task. Since VLM combines a Vision Encoder and an LLM Decoder, request volume and execution stages must be managed systematically when many cameras generate requests at once.

EVA does not run VLM on every frame. VM first checks object existence and baseline conditions, and VLM runs only when needed.

For example, in PPE non-compliance detection, frames without a person are filtered out early. Follow-up reasoning runs only when a person region is confirmed. Complex scenarios are also split into smaller tasks such as detection-stage decisions, exception checks, and alert-message generation, rather than handled as one large request.

This lets EVA manage VLM calls efficiently on NPU and concentrate compute resources on requests that actually require semantic reasoning.

5. Distributed Multi-NPU Workers and vLLM Instances​

To run EVA as a commercial service on NPU, the system must stably handle simultaneous Vision and VLM requests from many cameras. EVA therefore separated Vision and Agent domains, and distributed NPU cores and vLLM instances by role.

In the Vision domain, object-detection workers are mapped per NPU core so many camera requests can run in parallel. Worker counts are tuned by camera volume and per-model request load so NPU cores are used stably.

In the VLM domain, requests are not concentrated into a single vLLM instance. Multiple vLLM instances are distributed to secure concurrent throughput. This is especially important for balancing load in the Vision Encoder path.

In short, EVA's NPU optimization is not just about adding more NPU hardware. It is about placing Vision workers and vLLM instances according to NPU architecture to improve end-to-end inference pipeline stability.

6. NUMA Alignment and CPU Pinning​

NUMA stands for Non-Uniform Memory Access, a memory architecture where access latency varies by CPU socket and memory locality.

In NPU optimization, server resource placement is as important as models and pipelines. In real production environments, multiple vLLM instances run on a single server. Which CPU cores each process uses, which NUMA node those cores belong to, and how physically close they are to the NPU can all affect end-to-end response time.

In multi-instance environments, it is critical to align data movement paths across CPU, memory, and NPU clearly. For this reason, EVA validated a structure that applies CPU Pinning and NUMA Alignment per vLLM instance.

In final production environments, each vLLM should run in an isolated container or pod, with CPU Pinning and NUMA Alignment applied per deployment.

This is not simple server tuning. It is an infrastructure-level optimization required for stable NPU-based AI service operation.

Closing​

Throughout this optimization journey, EVA did not treat NPU as a simple GPU replacement. To run a GPU-centric AI service pipeline stably on NPU, we had to jointly optimize model structure, input pipelines, inference stages, worker placement, vLLM instance configuration, and server resource alignment.

The key elements optimized together for stable EVA operation on NPU were:

• NPU-aligned model structures
• Fixed input pipelines
• Crop-based input optimization to offset resolution-loss effects
• Detection-quality validation based on real operational data
• Distributed processing architecture across Vision and VLM
• Resource alignment based on CPU Pinning and NUMA Alignment

Through this process, EVA secured a structure that maintains performance in major detection scenarios while handling many concurrent camera requests stably in NPU environments.

Going forward, EVA will continue advancing Vision Encoder parallelization, model quantization, multi-NPU scheduling, and container-based resource isolation in Rebellions NPU environments, expanding into a Physical AI platform that runs reliably across diverse hardware infrastructures.

To operate EVA reliably for a specific purpose, key settings such as object targets, detection sensitivity, and vision models must be continuously tuned to each scenario and camera environment. The core challenge is that it is very difficult for humans to consistently decide "what value should be set now" by combining historical data with the current scene.

To reduce this operational burden and make desired detections easier and more accurate, EVA plans to apply Meta Agent. Meta intelligence will continue to evolve, and in v3.0, object-sensitivity recommendation and vision-model recommendation are provided first.

1. Why Meta Agent Is Needed More Than Ever​

In small environments, manual camera-by-camera tuning may appear manageable. But once the number of cameras exceeds 100, the operational reality changes completely.

Each camera has different installation height, field of view, lighting, background reflection, and workflow pattern. Even with the same scenario, false-positive/false-negative patterns vary by camera. At that point, per-camera optimization becomes necessary, and that work requires both significant effort and specialized operational know-how.

Typical pain points in real operations:

• If object definitions are too broad or ambiguous, false positives increase and unnecessary inference grows.
• If sensitivity is too low, non-target objects are detected; if too high, required targets are missed.
• Model performance varies by scenario (for example, fire/smoke/fall), camera angle, lighting, and background structure.
• Even after changing settings, effects are hard to validate immediately, and manual tuning cost rises sharply with camera count.

In short, one-time initial setup cannot sustain long-term detection quality. A data-driven auto-optimization layer is required.

2. How Meta Agent Operates​

Meta Agent periodically checks each camera state and operates based on the following signals:

• Recent detected-alert data
• Accumulated user feedback
• Accumulated image volume available for recommendation decisions

The key point is this: the more feedback a camera accumulates, the more frequently Meta Agent can analyze that camera, and the more accurate its recommendations become.

So Meta Agent is not a one-time analyzer. It continuously adjusts recommendation frequency and quality by reflecting both camera-level operating status and feedback accumulation.

As shown below, operators can view actual detection alerts together with recommendation messages, while EVA provides sensitivity-adjustment recommendations based on accumulated feedback and detection results. For example, when a specific object is repeatedly detected at a low sensitivity level, EVA can recommend adjusting to a sensitivity value that better fits the current scene.

This process does not end at recommendation alone. By continuously analyzing real detection data from live environments, it also improves EVA's operating strategy and recommendation precision over time.

As camera count grows and more data accumulates, EVA evolves toward better field fitness.

3. Object-Sensitivity Recommendation Logic​

Meta Agent recommendation logic follows this sequence:

• Run inference on the same object with five vision models.
• Perform automatic labeling by ensembling model outputs.
• Verify whether true/false detections can be separated on 20+ images.
• Generate sensitivity/model recommendations only when separability is confirmed.

Why ensemble instead of a single model:

• Objects falsely detected by a vision model are likely to propagate as false positives into VLM-stage decisions.
• If early visual recognition is unstable, higher-level reasoning can also be distorted.
• Therefore, establishing object validity through cross-model agreement is more stable than trusting one model output.¹²

There is also a clear efficiency benefit:

• VLM inference consumes substantial resources, so large-scale labeling by repeatedly calling VLM is inefficient.
• In contrast, multi-vision-model ensembling can validate large samples faster at lower cost, making it more practical for operations.

Internal validation showed that multi-model ensemble labeling achieved about 30% higher labeling accuracy than single-judgment labeling.

Because all vision models already infer on the same images during this process,

• appropriate sensitivity can be recommended immediately for the current model, and
• if all scenarios detect the same target, model-change recommendation can also be connected.

Recommendation message example:

False positives are occurring for detection target {object}. Try switching to model {A} and adjusting sensitivity to {0.xx}.
ℹ️ Model-change recommendation is provided only when all scenarios share the same detection target.

Closing​

Meta Agent is EVA's operational-optimization layer that shifts sensitivity/model tuning from intuition-driven manual work to data-driven recommendation.

Starting with object-sensitivity and vision-model recommendation in v3.0, Meta intelligence will continue to evolve to improve scenario-level operational quality over time.

In EVA v3.0, we introduced Thinking Mode to reduce false positives in safety incident detection for hazards that can be identified immediately from a single scene. The key idea is that the VLM does not jump to conclusions. Instead, it performs internal reasoning first, checks false-positive risks and logical inconsistencies from multiple angles, and then makes the final hazard decision.

Thinking Mode is especially suitable for scenarios where the hazard is immediately discernible on screen, but false-positive risk still needs multi-angle verification.

In addition, VLMs can show a tendency to align with user queries depending on context, and continuously adding exception rules in a base mode can increase latency and operational complexity.¹²

As a result, EVA v3.0 uses the model's internal Thinking process to quickly interpret scenes, evaluate false-positive possibility with common-sense reasoning first, and then issue alerts, improving overall detection reliability.

1. Why Thinking Mode Is Needed, and How It Improves Reliability​

In a simple query-driven baseline, even visually obvious cases such as "is there a fire?" or "did a person fall?" can still generate false positives due to camera angle, reflections, occlusion, or low-light conditions.

On top of that, a rule-patching approach that keeps adding exception rules whenever false positives are found has clear operational limits:

• Increasing scenario-management complexity as exception rules accumulate
• Potentially longer inference paths and higher latency
• Reduced operational consistency due to user-by-user rule variations

To address this, Thinking Mode is designed to prioritize "review before decision" rather than "quick assertion."

Core Principles​

• Focus on immediately discernible scenes: Prioritize incidents that can be judged clearly within a single frame.
• Multi-angle false-positive review: Use internal reasoning to check counter-evidence and logical contradictions first.
• Final alerts must match incident items: Trigger alerts only when detected evidence aligns with user-configured incident items.

Decision Guidelines​

Thinking Mode explicitly defines the following operating policies at the system-prompt level so the model does not drift into overly deep reasoning.

• Direct visual evidence first: Use only directly visible evidence from the main scene.
• Judge only active incidents: Evaluate only active incident items from user input, preserving order.
• Return False when ambiguous: If evidence is weak, noisy, occluded, or ambiguous, return False.
• Incident-specific precision guards:
  • Fall/down: Crouching, kneeling, sitting, or perspective ambiguity alone must not trigger True.
  • Smoke/spark: Steam, dust, reflection, blur, or lighting artifacts alone must not trigger True.
  • Fire: Bright light or reflection without direct flame/combustion evidence must not trigger True.
• Strict output normalization: Return per-incident outputs only in the required JSON schema for stable post-processing.

In short, Thinking Mode is not just a detector. It is an operational decision mode that combines evidence-centered judgment with false-positive suppression guards.

2. Performance Review​

Thinking Mode is evaluated not only by final right/wrong labels, but by how the agent produces intermediate evidence and matches that evidence against scenario rules. As shown below, the model first reasons over scene cues, then determines whether to alert by matching user-requested incidents.

Source: National Information Society Agency (NIA), Korea

This case corresponds to fall/down detection. In Thinking Mode, the model reaches alert: true after checking the following core reasoning points.

• Scene analysis: Detect a worker in PPE (white protective suit and hard hat) lying horizontally on the industrial floor.
• Posture judgment: Determine that the posture is not crouching/sitting/working posture, but loss of upright posture with body-on-ground evidence.
• Fall guard validation: Verify that the condition of direct visual evidence for collapse/body-on-ground is satisfied.
• Ambiguity check: Confirm low likelihood of benign alternatives (for example stretching or temporary posture change).
• Final decision: Sufficient direct evidence of a fall/down event, therefore alert: true.

In contrast, this case appears like a vehicle fire at first glance, but is actually not a fire. Thinking Mode still avoids single-cue decisions and reviews the scene from multiple angles.

Key checkpoints from intermediate reasoning:

• Scene-context check: Confirm nighttime parking-lot context with multiple strong light sources (streetlights/headlights).
• Re-interpret suspicious cue: Treat rear red glow as a possible reflection/light artifact instead of immediately labeling it as flame.
• Apply fire guards: Re-check direct flame shape, combustion signals, and smoke evidence.
• Final decision: No clear flame/combustion evidence, therefore alert: false.

This shows that even for visually confusing cues such as strong red illumination, Thinking Mode reviews counter-evidence before deciding, reducing false positives and delivering more reliable alerts.

As a result, the following scenario-level metrics were measured for Thinking Mode:

Fall/Down​

Smoke/Spark​

Fire​

In summary, Thinking Mode significantly improves performance for immediately discernible fall/down, smoke/spark, and fire scenarios.

⚠️ Thinking Mode can also perform well in other scenario types, but for cases requiring detailed human-action/gear-state inspection or strict environment-specific rule reasoning, inference time can become very long. We recommend using it selectively for suitable scenarios.

3. Usability: Expand Coverage by Editing Incident Items, Without Adding New Scenarios​

In EVA v3.0, Thinking Mode can expand immediately discernible incident detection by editing incident items, without creating a new scenario each time.

## 💡 Detection with Thinking Mode
When an object is detected, the agent reviews false-positive possibility and logical inconsistencies at the configured detection interval,
then decides whether a hazardous incident is present. If hazardous, an alert is triggered.

### Hazard Incidents (up to 3)
Complex or ambiguous incidents may require longer reasoning time.
Please define incidents that can be clearly judged within a single scene.

- Fire outbreak

**Good examples**
- Fire outbreak
- Smoke outbreak
- Person collapsed on the floor

**Not recommended**
- Hazard detection based on specific human actions
- Scenarios requiring composite object-state reasoning

Closing​

The core of Thinking Mode is "review before decision" rather than "immediate reaction." This helps lower false positives in immediately discernible incident scenarios and provides alert quality that operators can trust in real deployments.

In EVA v3.0, we introduced PPE Mode, a redesigned VLM inference pipeline to reduce false positives in PPE non-compliance detection. The key change is moving away from "directly answering user queries" toward an agent flow where the model first describes what it observes and then makes decisions based on that evidence.

1. Why did the previous approach produce many false positives?​

In PPE non-compliance detection, when user queries are highly directive (for example, "find people not wearing helmets"), some VLMs tend to generate affirmative, query-aligned responses. In these cases, even weak visual evidence can still lead to a "non-compliance" answer, increasing false positives.

This is closely related to known issues in VLM/LLM systems, including hallucination and sycophancy (overly aligning with user intent).¹²³

By contrast, prompts focused on description (for example, "describe what you see in the image") reduce pressure to agree with user intent and tend to produce more faithful object-state descriptions.

2. EVA v3.0 Agent Upgrade: From "Answering the Query" to "Describe, Then Decide"​

PPE Mode in EVA v3.0 separates detection into a 3-stage pipeline so that VLM decisions are less directly biased by user intent.

If the previous approach was close to "user query -> immediate decision," PPE Mode separates evidence formation into "target selection -> body-part verification -> state description -> rule matching" to reduce false alarms.

Core Principles​

• Check work context first: Select candidate workers using person-level boxes, while also incorporating full-image context for scenarios like elevated work.
• Separate wearing state from verifiability: Distinguish "what is worn" from "whether the required body part is actually visible."
• Keep final decisions rule-consistent: Standardize alert criteria by matching required items, synonyms, and required-body-part visibility together.

3-Stage Flow​

Stage 1. Candidate Worker Selection and Required Body-Part Definition​

• Enrich Required Equipments
  • Extract synonyms for required detection items.
• Find Worker
  • For each person box detected by the object detector, determine whether that person matches the target work situation.
  • Since some scenarios (for example, elevated work) require full-scene context, the decision also uses the full image.
  • If multiple candidates exist, they are processed in parallel.
  • ℹ️ If there are 5 or more people, only the top 5 largest bounding boxes are evaluated.
  • If at least one relevant worker is found, proceed to Stage 2.
• Required Body Parts
  • Derive body parts needed to verify each required item.
  • Example: mask -> frontal/side face, helmet -> head

Stage 2. Worn-Item Extraction and Body-Part Check​

• Explain Worker
  • Extract worn items (helmet, mask, etc.) for workers detected in Stage 1.
  • If multiple workers are detected, run in parallel.
• Check Body Parts
  • Verify whether required body parts are actually visible for workers detected in Stage 1.
  • If multiple workers are detected, run in parallel.

Stage 3. Rule Matching, Alerting, and Description Generation​

• Matching Equipments
  • Trigger an alert when a detected item does not match the required-item synonym set and the required body part is visible.
  • Include missing required items in the alert message.
  • Example: PPE non-compliance detection (helmet, mask)
• Image Description
  • Generate an image description together with the alert.

With this structure, PPE Mode avoids "immediate VLM assertions" and instead makes final decisions through "target selection + body-part check + item extraction + rule matching," improving reproducibility and operational trust.

3. Performance Review​

PPE Mode is not just evaluated by final right/wrong labels. It operates by generating intermediate evidence and matching it against scenario rules. As shown below, it first infers worn items and verifiable body parts, then determines whether to trigger alerts by matching with user scenarios.

For an image like this, the agent generates intermediate outputs such as:

# Worn items
"worn_items" = ["mask", "gloves", "pants", "shoes"]
"evidence" = "The person is wearing a mask, gloves, pants, and shoes."

# Required body-part check
"target_region": "upper head and crown area",
"visible": true,
"evidence": "The top of the person's head and crown are clearly visible from the front."

PPE Mode then matches these outputs to the user scenario (for example, missing mask and helmet), and triggers an alert when the condition required item not detected + required body part visible is satisfied. In this example specifically, the person is detected with a mask but without a helmet, and the head region is visible, so a helmet-missing alert is generated.

As a result, across a 7-scenario PPE dataset, we observed the following performance gains:

Numerically, PPE Mode shows a clear improvement in detection reliability over the base mode.

Operationally, the most meaningful change is the large increase in precision. That means a higher portion of "non-compliance" alerts correspond to actual violations, directly reducing unnecessary follow-up checks and alert fatigue.

4. Usability: Change Detection Targets Without Adding New Scenarios​

PPE Mode generates detection scenarios in the following format, so users can switch required PPE targets without creating new scenarios each time.

## 💡 PPE mode is enabled for detection.
When a person is detected, the agent checks whether the person matches the configured work situation at the configured detection interval.
If any required item is not satisfied, an alert is triggered.

> ⚠️ For accurate PPE verification, configure detection targets to include the full body in the person box.

> Example: "person" or "worker"

> If there are 5 or more people, detection runs only for the top 5 largest boxes.

### Work Situation (only one can be specified)
Describe the work state or action of the target person.
- Seated and performing soldering work


### Required Items (up to 3)
Describe the PPE items that must be worn.
- Mask
- Helmet

Closing​

The core of PPE Mode is the shift from "answer first" to "reason from evidence first." By doing so, we focus on reducing false positives in PPE non-compliance detection while improving alert quality that operators can trust in real environments.

As a next step, we will add more real field image cases and publish both quantitative and qualitative analyses on where improvements are most significant.

Vision Transformers (ViTs) power a wide range of tasks, from classification and detection to segmentation and multimodal backbones. But their high compute cost often becomes a deployment bottleneck.

In this post, we walk through the ICML 2026 ToaSt paper in detail: the motivation, method design, and key experimental results.

The core idea of ToaSt can be summarized in one line:

• Decouple MHSA and FFN compression, optimize them with different strategies, and improve the accuracy-efficiency trade-off while avoiding cross-layer propagation issues.

1. Background: Where ViTs become expensive​

ViT compute mainly comes from two sources:

• Attention: roughly (O(N^2)) with token length (N)
• FFN: heavy channel-wise computation around hidden dimension (D)

As highlighted in the paper, a standard ViT spends about 61% of FLOPs in FFN and around 19% in attention. This means attention-only acceleration is not enough; FFN redundancy must be addressed directly.

Figure 1. ToaSt decouples MHSA and FFN compression to avoid harmful cross-layer propagation.

2. Limits of prior approaches​

The paper groups prior ViT acceleration methods into three categories.

2.1 Structured Weight Pruning​

• Strength: removes heads/channels/blocks in hardware-friendly structured form
• Limitation: often requires long retraining to recover accuracy
• Practical issue: ViTs are already expensive to train, so long post-pruning fine-tuning is costly

2.2 Token Compression / Token Merging​

• Strength: directly reduces attention cost by shrinking (N)
• Limitation 1: does not directly target dominant FFN (D^2) complexity
• Limitation 2: token decisions propagate globally to later layers, making optimization harder

2.3 Joint / Hybrid Methods​

• Strength: can optimize multiple axes at once
• Limitation: more coupled optimization and higher tuning complexity
• Some approaches are also more hardware/kernel dependent in practice

ToaSt takes a different route: not one large coupled optimization, but module-wise decoupled compression.

3. Method: ToaSt as Layer-Independent Compression​

ToaSt preserves each block interface ((N \times D)) while compressing internal computation. This reduces the cascading side effects caused by global structural changes.

3.1 MHSA: Coupled Structured Pruning​

In MHSA, Q/K/V/Proj are mathematically coupled. ToaSt enforces synchronized index pruning rather than pruning them independently.

• Q-K synchronized pruning
• V-Proj synchronized pruning
• Reduce per-head internal dimension (d_k), while preserving the global interface dimension (D)

Importance is computed with a geometric-median-based criterion. Except for small-model settings, the first layer is typically preserved and later layers are pruned aggressively. The paper reports that aligned (coupled) pruning significantly mitigates accuracy collapse compared with non-aligned pruning at high pruning ratios.

3.2 FFN: Token Channel Selection (TCS)​

FFN has a (D \rightarrow 4D \rightarrow D) structure and dominates total FLOPs. ToaSt introduces training-free dynamic channel selection (TCS) for FFN.

The paper’s FFN analysis reports three consistent patterns:

• increasing activation sparsity in deeper layers
• collapsing effective rank
• high (R^2) reconstruction, indicating strong linear channel redundancy

Based on this, TCS samples a subset of tokens, estimates channel importance, and keeps only informative channels. Its importance metric combines global CLS-driven context and patch saliency; for CLS-free architectures (e.g., Swin), patch-only scoring is used.

The pruning policy is asymmetric:

• FC1 (expansion) is pruned conservatively
• FC2 (reduction) is pruned more aggressively in deeper layers (up to 90% in the reported setup)

Figure 2. Swin-Base FFN analysis: redundancy increases in deeper layers.

Figure 3. ToaSt overview: coupled MHSA pruning + FFN TCS.

4. Experimental setup and key results​

4.1 Setup​

• Classification: ImageNet-1K
• Downstream transfer: COCO 2017 detection (Cascade R-CNN / Mask R-CNN pipelines)
• Backbones: 9 models across DeiT (T/S/B), ViT-MAE (B/L/H), Swin (T/S/B)
• Metrics: Top-1/Top-5, FLOPs, and throughput/speedup on H100

4.2 ImageNet results​

The main message is simultaneous compute reduction and accuracy gain.

1. ViT-MAE-Huge: Top-1 88.52% (vs. 86.88 baseline, +1.64%p), 39.4% FLOPs reduction, 1.59x throughput
2. DeiT-Small: Top-1 83.40% (vs. 79.82 baseline), 45.7% FLOPs reduction, 2.07x throughput
3. Swin-Base: Top-1 85.21% (vs. 83.50 baseline), 42.7% FLOPs reduction, 1.28x throughput

At similar FLOPs budgets, the paper reports multiple cases where ToaSt outperforms token-compression baselines such as ToMe and DiffRate.

4.3 Downstream transfer (COCO)​

Compressed backbones remain competitive when transferred to detection:

• Swin-Small: 52.2 box mAP (baseline 51.9)
• Swin-Base variants: 52.2 / 51.8 box mAP

This suggests ToaSt removes architectural redundancy rather than only overfitting classification behavior.

5. What the ablations show​

The ablations separate the contribution of each component:

• MHSA-only: often improves speed but can hurt accuracy
• MHSA + TCS (full ToaSt): adds more speed and recovers (or exceeds) baseline accuracy

The FC1/FC2 sensitivity analysis also supports the asymmetric pruning policy:

• FC1 is more sensitive in early layers
• FC2 tolerates aggressive pruning in later layers

The paper interprets this as a sign that TCS filters redundant channel noise and can behave like implicit regularization.

6. Practical takeaways​

From an engineering perspective, ToaSt is attractive because:

• decoupled modules simplify optimization
• FFN-focused reduction targets the dominant compute cost
• structured outputs are hardware-friendly on commodity GPUs
• larger models appear to need fewer recovery epochs after pruning

The reported inverse scaling trend in recovery epochs is especially interesting for large foundation backbones.

7. Limitations and future work​

The paper explicitly notes one current limitation: layer-wise pruning ratios are manually tuned. Future directions include:

• learnable/automatic ratio optimization
• extension to VLM settings
• combination with quantization

Conclusion​

ToaSt addresses two recurring ViT compression pain points at once:

• token-only compression is insufficient for FFN-dominant compute
• globally coupled pruning often increases retraining cost and instability

By decoupling MHSA and FFN compression and tailoring each to its own structure, ToaSt achieves a strong and consistent accuracy-efficiency trade-off across model families.

The key message is simple:

For ViT acceleration, token reduction alone is not enough. Channel redundancy in FFN must be addressed, and decoupled module-aware design is a practical way to do it.

One of the core features of the EVA service is real-time streaming, which allows users to monitor video feeds from dozens of cameras simultaneously. As usage expanded from brief checks to long-term on-site monitoring, unexpected performance bottlenecks began to surface.

"When I leave the screen on for a long time, the browser gradually slows down and eventually the tab crashes."

To address this issue, we’d like to share our journey of improving the rendering architecture using Canvas, Web Worker, and OffscreenCanvas.

1. Background: Why Did Problems Appear Over Time?​

In its early days, EVA adopted a very common streaming approach using the <img> tag combined with Blob (Object URL).

Previous Approach (Blob-Based Rendering)​

1. MJPEG stream data is received from the server in Blob form.
2. A temporary URL is generated using URL.createObjectURL(blob).
3. The URL is assigned to the src of an <img> tag, allowing the browser to render the image.

While this implementation was simple, two critical issues emerged in the specialized environment of long-term monitoring.

• Memory Overhead: A unique URL string is generated for every frame (around 30 frames per second). Even when calling revokeObjectURL, delays in the browser’s internal image cache and garbage collection (GC) caused memory usage to continuously increase, eventually leading to Out of Memory (OOM) errors.
• Main Thread Blocking: Image decoding occurs on the main (UI) thread. When processing high-resolution video, the event loop is delayed, resulting in UI lag such as slow clicks or scrolling—commonly known as jank.

2. Network Tab Analysis: Understanding MJPEG​

The first step in improving performance was analyzing the network layer. MJPEG streaming behaves differently from typical HTTP requests.

multipart/x-mixed-replace​

MJPEG uses the Content-Type: multipart/x-mixed-replace; boundary=... header, which allows the server to continuously push image frames over a single HTTP connection.

• Network Tab Characteristics: The request never completes and remains in a 'Pending' state. The browser keeps the connection open and continuously receives binary data.
• Binary Data Structure: Each frame consists of JPEG binary data (0xFF 0xD8 ... 0xFF 0xD9) separated by a specific boundary string.

Because the previous approach converted this massive stream of binary data into Blobs and parsed it on the main thread, browser load increased exponentially as more data accumulated.

3. First Optimization: Canvas and createImageBitmap​

To move away from memory management that relied heavily on the browser’s garbage collector, we introduced the Canvas API and adopted an explicit memory management approach.

Asynchronous Bitmap Rendering​

The createImageBitmap API allows images to be decoded asynchronously in the background before being rendered to the screen.

// @src/entities/devices/components/stream/MJPEGStream.tsx
// Immediately release memory after drawing the bitmap on the canvas
const bitmap = await createImageBitmap(blob);
ctx.drawImage(bitmap, 0, 0);
bitmap.close(); // Explicitly release memory

The key point of this approach is bitmap.close(). By explicitly destroying bitmap resources after use, we were able to keep memory usage stable. In addition, by eliminating reflow caused by changing the src of an <img> tag and switching to GPU-accelerated canvas drawing, overall rendering efficiency was significantly improved.

4. Second Optimization: Separating Computation with Web Workers​

While rendering became lighter, the task of receiving stream data and extracting JPEG frames from binary data (boundary parsing) was still handled by the main thread. Performing real-time string searches on millions of bytes per second places a heavy burden on the CPU.

To solve this, we introduced Web Workers and applied a clear division of responsibilities:
"Data processing in the background, rendering on the main thread."

Optimizing Data Transfer (Transferable Objects)​

When sending large images from a worker to the main thread, copying data results in severe performance degradation. We leveraged Transferable Objects to transfer ownership of memory without copying, enabling a zero-copy data flow.

5. Final Optimization: Introducing OffscreenCanvas​

Despite these improvements, the final drawing step still occurred on the main thread. The final piece of the puzzle was OffscreenCanvas, which allows control of the canvas itself to be transferred to a worker.

Even when the main thread is blocked (left), image processing running in the worker continues to update in real time without interruption. (Source: Kakao Tech Blog)

Toward 0% Rendering Load on the Main Thread​

After transferring control using transferControlToOffscreen(), rendering is performed entirely inside the worker.

// @src/entities/devices/components/stream/mjpeg.worker.ts
const bitmap = await createImageBitmap(blob);
if (ctx && canvas) {
  // Worker directly draws on the canvas (0% main-thread interference)
  ctx.drawImage(bitmap, 0, 0);

  if (config.showArea && config.area) {
    drawPolygonArea(ctx, config.area); // Area overlay logic also runs in the worker
  }
}
bitmap.close();

With this architecture, no matter how heavy the workload on the main thread becomes, streaming video continues to play smoothly and independently on a separate thread.

🌐 Browser Compatibility and Automatic Fallback​

While OffscreenCanvas offers powerful capabilities, browser support varies. In the EVA service, browser features are automatically detected and conditionally handled based on the user’s environment.

EVA’s Adaptive Rendering Strategy:

• Modern browsers: Enable OffscreenCanvas to keep main-thread load at 0%.
• Older browsers (e.g., Safari 15 or below): Detect feature availability and automatically fall back to the first optimization approach—main-thread Canvas rendering.

This ensures a seamless streaming experience across all browser environments.

6. Additional Optimizations: Buffer Reuse and Faster Parsing​

Performance is determined by details. We applied several additional optimizations within the worker logic.

1. Fixed Buffer Reuse: Instead of creating new Uint8Array instances each time, we reused fixed-size buffers and managed data using copyWithin. This significantly reduced the frequency of garbage collection (GC).
2. High-Speed Parsing with indexOf: Rather than using simple loops to find matching bytes in binary data, we leveraged the built-in indexOf method to skip unnecessary byte comparisons. Even this simple optimization dramatically reduced frame drops.

7. Conclusion: A More Robust EVA Monitoring Environment​

Through this optimization effort, the EVA service achieved the following results:

• Memory Stability: Memory usage remains stable even during long-term operation, eliminating OOM errors.
• UI Responsiveness: UI interactions such as menu navigation and button clicks remain smooth—even during high-resolution streaming—at a near-native app level.
• Stable Frame Rates: By separating threads, consistent frame rates are maintained regardless of network latency or main-thread load.

This project reinforced a key principle of frontend performance optimization:
"How free you keep the browser’s main thread makes all the difference."

Thank you for reading!

Technology Summary​

• Web Workers API: Execute computations on background threads
• OffscreenCanvas: Rendering independent of the main thread
• createImageBitmap: Asynchronous image decoding with explicit memory management
• Transferable Objects: High-speed data transfer without copy overhead

References​

• Kakao Tech Blog - OffscreenCanvas
• MDN - Web Workers
• MDN - OffscreenCanvas
• MDN - createImageBitmap
• MDN - Transferable Objects

EVA uses AI Agents as core executors not only in development, but also throughout the requirement management process.

When a problem needs to be solved or a new improvement idea emerges in the field, the process begins simply by sending an email to eva-req@spectrabrain.ai.

What matters here is that writing the email itself does not need to be complicated.

Requesters do not need to spend time formatting their requests. Instead, they can briefly describe the pain points they experienced and the improvements they expect.

From that point forward, automated Agent workflows take over the refinement, analysis, and prioritization process.

1) Even Simple Emails Are Structured by Agents​

Requesters focus on context rather than format when sending requirement emails.

The Review Agent transforms these free-form inputs into structured requirements that can be directly reviewed by development teams.

During refinement, the Agent supplements missing context and converts ambiguous expressions into actionable language suitable for technical review.

In other words, this step transforms human-friendly input into a system-readable requirement structure.

At this stage, the requirement is no longer just a note—it becomes an organized unit that supports implementation review and priority assessment.

2) Analysis Based on EVA Manuals and Logic Documentation​

Once refined, the requirement is analyzed together with EVA’s internal knowledge base.

This includes:

• User Manuals
• Technical and Logic Documentation

Using these documents, the Review Agent performs an initial analysis to identify affected areas, possible solution paths, and implementation priorities.

The analysis covers:

• Impacted features and logic
• Whether the issue can be resolved with existing functionality
• Whether new implementation is required
• Technical risks and expected impact
• Development priority

Through this process, a requirement evolves beyond simply describing what the problem is.

It becomes an executable development unit that explains how it can be solved and why it should be addressed now.

3) The Agent Loop Continues After Release​

The workflow does not stop after implementation.

Once a release is completed, the Release Agent updates the manuals and logic documentation based on the latest changes.

These updated documents then become the knowledge base for future requirement analysis, allowing Agents to make increasingly accurate decisions as the product evolves.

The key point in EVA’s requirement operations is that this process is continuous.

Requirement collection, analysis, development, release, and documentation updates are all connected as a single automated loop without fragmentation.

Through this loop, Agents continuously learn from signals such as:

• Which requirements were actually implemented
• How they were resolved
• Which original expressions were unclear or inaccurate

The Review Agent structures inputs, analyzes them, and proposes priorities.

The Release Agent reflects implementation results back into documentation, enabling more accurate analysis for future incoming requirements.

In practice, this creates meaningful improvements such as:

• Higher requirement analysis accuracy
• Automatic detection and cleanup of duplicate requests
• Improved documentation quality

Conclusion: Agents Are Not Features, but Operational Processes​

In EVA, Agents are not just assistive features for isolated tasks.

From the moment a requirement is submitted by email, Agents refine the content, analyze it based on manuals and logic documentation, derive priorities, and update documentation again after release.

In other words, EVA’s requirement management is not a manual process where people repeatedly format requests and organize follow-up work.

It is a workflow where AI Agents manage the entire lifecycle, structure information into maintainable forms, and accumulate knowledge for future decision-making.

As a result, requirements are no longer consumed as one-time requests.

They accumulate as operational data that helps define the product’s evolution more accurately.

EVA uses Agents not simply to automate tasks, but to turn requirement management itself into a continuously learning and improving operational process.

Recent data center fires have gone beyond physical damage, leading to massive financial liabilities due to service disruptions.

👉 SK C&C Pangyo Data Center Fire (2022): A fire originating in a lithium-ion UPS battery room caused major service outages, including Kakao, with estimated damages reaching trillions of KRW.

👉 OVHcloud Fire in France (2021): Triggered by UPS power equipment, this incident resulted in approximately €105M in total damages, with €58M covered by insurance—significantly increasing insurer exposure.

These large-scale incidents highlight that modern AI-era data centers carry risks that can no longer be controlled with traditional physical security measures alone.

😲 2. Data Center Insurance Structure and the Surge in AI GPU Center Premiums​

Data center insurance is typically structured as a bundled package including:

• Property (buildings and servers)
• Business Interruption
• Cyber Liability
• General Liability

👉 Premium Rates Based on Asset Value Property insurance premiums usually range from 0.2% to 0.5% of total asset value. However, AI data centers are now facing rapidly increasing premiums due to elevated risk classifications.

👉 Risks of High-Density Servers AI GPU clusters have significantly higher power density compared to traditional servers, directly increasing fire risk.

😎 3. Key Underwriting Checklist from Insurers​

Global insurance brokers (Aon, Marsh, FM Global, etc.) evaluate risk based not on facility size, but on technical measures that reduce incident probability. EVA provides strong advantages across these evaluation criteria.

👉 Power Infrastructure Risks
Current Status: 40–50% of data center fires originate from electrical systems. Lithium-ion battery thermal runaway is a major driver of rising premiums.
🌈 EVA’s Role: Detects minute temperature variations and thermal anomalies at the battery cell level in real time, significantly reducing lithium battery risks.

👉 Advanced Fire Detection and Suppression Systems
Current Status: Early smoke detection and gas-based suppression systems are top underwriting priorities.
🌈 EVA’s Role: AI-powered visual intelligence enables instant detection of flames and smoke, reducing detection time to seconds.

👉 Operational Risk (Human Error)
Current Status: Insurers heavily assess 24/7 monitoring systems and thermal inspection practices.
🌈 EVA’s Role: Transforms manual, human-dependent inspections into automated AI-driven monitoring.

❣️ 4. Economic Impact of EVA Adoption: A Risk Engineering Approach​

The insurance market is shifting from post-incident compensation to proactive risk engineering—reducing the likelihood of future incidents. AI safety solutions like EVA create a win-win structure for both insurers and policyholders.

👉 Direct Insurance Premium Reduction (15%–30%) Well-implemented risk management systems can lead to premium reductions of 15% to 30%. For data centers worth hundreds of billions of KRW, this translates into direct financial benefits that exceed the cost of deploying the solution.

👉 Five Core Risk Control Points EVA addresses the key underwriting factors that directly impact insurance premiums:

• Continuous UPS battery monitoring: Early detection of thermal runaway
• High power density 대응: Focused monitoring of cable and PDU overheating in AI GPU environments
• Intelligent fire detection: Ultra-fast alerts based on visual data
• 24/7 uninterrupted monitoring: Detection of unsafe behavior and human error
• Faster incident response: Reduced time from alert to action

🥰 5. AI Safety Systems Define Data Center Business Continuity​

From a data center operator’s perspective, EVA is not just a “security CCTV system.”

• Financial Value: Reduces insurance premiums and prevents large-scale business interruption losses
• Operational Value: Establishes standards for managing power and fire risks in high-density AI environments
• Reputational Value: Strengthens brand trust as a “safe data center” validated by strict insurance assessments

Through a virtuous cycle of AI Safety System → Risk Reduction → Insurance Discount, data centers can achieve both maximum safety and economic efficiency.

Today, we would like to share the porting process of moving GPU-based models to NPUs, along with the technical challenges behind it, which many people have been curious about.

1. The NPU Porting Process for GPU Models​

Since NPUs are designed to accelerate specific types of computations, newly released models cannot be executed immediately without adaptation. To fully utilize the hardware’s capabilities, several essential steps are required.

• Model Conversion

  The original models developed in PyTorch or TensorFlow must be converted into an executable format that the NPU can understand. Using the ATOM Compiler from Rebellions, the model’s computational graph is analyzed and converted into the .rbln executable format optimized for the NPU architecture.

• NPU-Optimized Compilation

  The model is compiled into a hardware-optimized executable using the compiler in the Rebellions SDK (RBLN SDK).

  • Graph Optimization: Removes redundant operations and reorganizes the data flow.
  • Operator Fusion: Combines multiple small operations into a single large kernel to reduce memory access and execution overhead.
  • Data Layout Optimization: Adjusts tensor layouts to match the NPU memory architecture, improving data access efficiency.

• Quantization

  Computational precision is adjusted to match the NPU architecture, improving both performance and memory efficiency. In the case of EVA, we optimized the model to ensure stable performance under an FP16-based inference environment.

• vLLM Integration and Validation

  The optimized model is deployed within the vLLM-RBLN serving framework. Key metrics such as TTFT (Time To First Token) and throughput are measured and validated against GPU-based environments.

2. EVA Application Optimization and Technical Challenges​

After porting the foundation model, the next step is deploying the actual service layer—the EVA Application. During this stage, we have been implementing the following optimization roadmap.

• EVA Vision Optimization (1:1 Mapping & Batching)

  We mapped NPU cores and Vision Workers in a 1:1 configuration, eliminating context-switching overhead. In addition, by applying continuous batching techniques, we are building a foundation capable of processing data from hundreds of cameras in real time without latency.

• EVA Agent Optimization (Reducing VLM Load)

  The input resolution of the Vision-Language Model (VLM) was standardized to 1280×720, and a two-stage reasoning architecture was applied to minimize unnecessary VLM calls. This immediately reduces the computational load on the Vision Encoder, which is one of the most expensive components in the pipeline.

• System Memory Management and KV Cache Optimization

  In collaboration with Rebellions, we analyzed the memory usage patterns of vLLM-RBLN instances and improved resource utilization using a page-based memory management structure. This optimization allows the system to process a larger volume of visual data reliably within the same hardware environment.

• Parallel Processing of the VLM Vision Encoder

  We are also improving the parallel execution architecture of the Vision Encoder, which accounts for a large portion of the computation in VLM inference. By optimizing how Vision Encoder operations are distributed across multiple NPU cores, we aim to significantly improve VLM serving throughput.

3. Conclusion: Evolving from PoC to a Production-Ready Solution​

We are continuously addressing technical challenges discovered during stress testing while refining optimizations that maximize hardware utilization. From parallel processing of the Vision Encoder through close collaboration with Rebellions to the development of an intelligent scheduler within the EVA platform, every step is part of transforming “EVA on NPU” from a simple proof-of-concept (PoC) into a production-ready solution.

Ultimately, the success of AI services depends on meeting three essential conditions: economic efficiency, scalability, and service quality. EVA will continue to actively adopt the latest NPU technologies and present a global standard for Physical AI platforms—delivering the most competitive TCO and outstanding performance for our customers.

Recently, Vision-Language Models (VLMs) have demonstrated exceptional performance in understanding individual images. Large-scale multimodal models have theoretically expanded the possibilities of multi-frame reasoning by introducing architectures that process multiple images alongside text prompts.

However, real-world industrial detection scenarios are far more complex than controlled research environments. Problems that seem straightforward with a single frame often lead to various false positives and edge cases in production.

Consider a scene where a person is lying on the floor. Looking at that single moment, it is easy to categorize it as a "collapse." But what if the previous frame showed them stretching, or simply changing posture while working?

In nighttime environments, lens flares, light reflections, or glare can mimic the color patterns of fire, leading to false fire detections when based on a single image. When even humans find it difficult to be certain from a single snapshot, providing a model with only one frame inevitably creates structural limitations.

These cases all share a common problem: a "lack of context."

Time is the Most Powerful Context​

Many detection scenarios inherently rely on a temporal flow.

For instance, "loitering" can only be defined by observing a pattern of staying in the same space for a certain period. Similarly, "long-term abandonment" requires the condition that an object remains unchanged for a specific duration after being placed.

Attempting to solve these problems with a single frame is structurally difficult because the focus must be on "change," not just "state."

We have categorized this into three levels of context:

• Single Image-based Judgment
• Short-term Multi-image Contextual Judgment (Momentary context)
• Temporal Judgment (Involving long-term flow)

In actual operating environments, these three levels coexist. Some scenarios are sufficient with a single frame, some require consecutive frames at intervals of a few seconds, and others require tracking a flow over tens of seconds.

EVA's Multi Frame Manager​

In EVA, user-defined scenarios are not treated as simple text conditions. The system analyzes the "level of context" required by each scenario and determines an appropriate frame collection strategy.

For example, "fainting detection" requires multi-images covering a few seconds before and after the event, rather than a single frame. In contrast, "long-term abandonment" requires continuous frame collection over a specific duration based on a sliding window.

The module responsible for this process is the Multi Frame Manager. This module dynamically determines the following based on the scenario characteristics:

• Number of frames required
• Collection intervals
• Retention time
• Event trigger expansion

Collected images are not simply listed. They are delivered to the VLM in a clearly sorted chronological order, accompanied by system prompts that guide the model to compare changes between frames.

Multi-Image Based VLM Inference Strategy​

When multi-frame input is received, the VLM does more than just return independent detection results. In EVA, we designed the inference structure to interpret multi-images as a continuous temporal context rather than an independent set of images.

To achieve this, frames are delivered to the model using the following strategies:

• Chronological Frame Alignment: Constructs time-series data from past to present to understand causality.
• Comparative System Prompts: Uses instructions like "Identify changes compared to the previous frame" to analyze inter-frame correlations.
• Temporal Reasoning: Derives logical conclusions based on state changes over time rather than fragmented snapshot judgments.

Case Study: The Power of Temporal Context in Reducing False Positives​

The following case demonstrates how fragmented information from a single frame is accurately corrected through the "context" of multiple frames.

• Single Image: A person is stationary in a low, prone position. A VLM looking only at this moment is highly likely to misinterpret the situation as "Collapse."
• Multi-Image: In the subsequent frames, subtle movements are captured—the person moves their arms to operate a phone and tilts their head to look at the screen.
• Result: Through Temporal Reasoning, EVA correctly concludes this is "Sitting and using a phone detected".

The core idea is to guide the model to understand the situation by comparing differences between frames, rather than judging each frame individually.

For high-risk detections like fainting, the model undergoes a process of Progressive Situation Refinement:

1. Initial State Identification: Identifying the target object and initial visual features (e.g., prone posture).
2. Dynamic Change Detection: Tracking meaningful changes in body angles or voluntary movements compared to previous frames.
3. Consistency Verification: Determining if the posture is a forced freeze due to impact or involves intentional actions.
4. Final Context Determination: Distinguishing between visual noise with similar patterns and actual events.

This Temporal Reasoning structure significantly reduces false positives in edge cases that plague single-image systems, providing much more stable results in real-world operations.

Ultimately, EVA’s multi-frame inference structure is not just about increasing the number of input images—it is an approach that directly integrates temporal change into the model's reasoning process.

The Cost of Multi-Frame: Computational Overload​

Improvements in accuracy come with a price.

While multi-frame reasoning allows for more visual information, it also leads to increased computational costs. In multimodal models, image inputs are generally converted into embeddings via a Vision Encoder before being passed to the LLM, a process that is relatively resource-intensive.

Specifically, multi-frame analysis often encounters the following:

• Identical or very similar images repeating in a sequence.
• Multiple requests referencing the same camera frame.
• Multiple queries performed on the same set of images.

In these cases, if the Vision Encoder processes the same image repeatedly, it creates unnecessary overhead.

In EVA, we developed a structure that maximizes the Encoder Cache feature provided by vLLM to solve this. vLLM offers an Encoder Cache Manager that allows the system to cache and reuse Vision Encoder results during multimodal processing.

• https://docs.vllm.ai/en/latest/api/vllm/v1/core/encoder_cache_manager/

By leveraging this, we can reuse previously generated encoder embeddings for identical image inputs, eliminating the need to repeat Vision Encoder operations. EVA applies a request management structure at the Agent Layer to effectively utilize this caching.

The Agent coordinates requests in the following ways:

• Organizing requests so that identical image inputs can be reused.
• Managing requests based on image units to enable cache hits.
• Optimizing request flow to prevent redundant encoding.

This allows us to minimize Vision Encoder operations and utilize GPU resources more efficiently, even in a multi-frame analysis environment.

Conclusion​

Multi-frame based VLM inference is an approach that significantly improves situational understanding and detection accuracy compared to single-image analysis.

However, as the number of frames increases, the computational load on the Vision Encoder grows significantly. Therefore, it is crucial to design a system that balances performance gains with computational efficiency and infrastructure costs.

EVA addresses this by actively utilizing vLLM's Encoder Cache and managing requests through the Agent Layer. Through this architecture, we maintain high inference performance while reducing unnecessary computations, continuously improving GPU efficiency and infrastructure operating costs.

This feature is available starting from EVA v2.6.0.

The competitive edge in AI is no longer defined by “how large or powerful a foundation model is,” but rather by “how effectively that model can perform complex tasks in real-world applications.”

A Paradigm Shift: From Performance to Execution​

• Old Paradigm: “How intelligent is it?” Until now, the AI industry has focused heavily on the scale and performance of foundation models. Large language models such as GPT-4, Claude, and Gemini competed on parameters, dataset size, and benchmark scores. The central question was: “How smart is the AI?”

• New Paradigm: “How much work can it actually perform?” OpenClaw introduces a fundamentally different question: “How effectively can the model perform complex tasks in real-world environments?” AI value is no longer measured by raw intelligence alone, but by its ability to execute within real computing environments.

The OpenClaw Approach​

• Real-time screen interpretation and contextual awareness The ability to interpret a user’s screen in real time shows that AI can move beyond text processing to understand and respond to visual context. This represents a practical implementation of multimodal AI.

• Direct application control The most innovative aspect is that AI directly controls various applications. This demonstrates AI’s evolution from a passive advisor or information provider into an active executor of real tasks.

EVA’s Approach: Execution-Centered AI in Physical Environments​

EVA (Evolved Vision Agent) proves the same value of execution in the harsh realities of industrial environments.

• Real-time site interpretation and visual reasoning Through CCTV streams, EVA understands on-site context. It goes beyond object detection to answer higher-level questions such as: “Why is that worker in danger?” or “Should a person be in that zone right now?” This is a multimodal AI service built by optimizing Vision-Language Models (VLMs) for industrial environments.

• Direct physical response and control EVA triggers physical actions in hazardous situations. Depending on severity, it can notify responsible personnel, activate on-site sirens, or even halt dangerous equipment processes. AI does not stop at judgment — it functions as the final executor that prevents accidents.

Industry-Wide Impact: A Major AI Paradigm Shift​

The direction of AI development is entering a fundamentally different trajectory. In the past, progress meant building larger models with higher benchmark scores. Now, practical utility has taken center stage.

Beyond creating intelligent models, the key metric has become how completely AI can accomplish tasks within real user environments. In other words, rather than model intelligence alone, the new compass for AI development is how deeply AI integrates into user workflows and delivers tangible value.

This paradigm shift is also redefining how we evaluate AI companies’ competitiveness. Moving beyond technological showmanship, market leadership will be determined by three core factors:

• Execution reliability Even highly intelligent systems must perform complex, multi-step tasks without interruption or error. Reliability will define trust.

• Integration capability AI must seamlessly interoperate with existing software and systems rather than exist in isolation.

• User experience (UX) Ultimately, success depends on how intuitively AI improves efficiency in real workflows and reduces user fatigue — not on technical flashiness.

The Dawn of the Execution-Centered AI Era​

The direction presented by OpenClaw represents more than technological advancement — it signals a paradigm shift across the AI industry. AI’s value will no longer be measured by how intelligent it is, but by how useful it is in accomplishing real work.

This transformation presents new opportunities and challenges for developers, enterprises, and users alike. Future AI competition will be decided not by benchmark scores, but by performance in real-world environments — marking a crucial turning point where AI truly becomes a tool that improves human life.

In this post, inspired by the recent text-to-video retrieval research Q₂E (Query-to-Event Decomposition)[1], we introduce Scenario Decomposition, a technique that enables VLMs to deeply understand complex, multi-scenario situations within a single frame.

1. 🚀 The Problem: Cognitive Bottlenecks in VLMs​

EVA Agents are required to monitor complex urban roads and multi-use facilities 24/7. In the early stages, we trusted the general-purpose capabilities of VLMs and expected them—like humans—to grasp all situations at once.

[Initial Approach: Unified Query] "Look at this CCTV footage and check whether there is a fire, a fallen person, a traffic accident, or an emergency vehicle entering."

In practice, however, the model revealed the limits of selective perception. For example, it may clearly recognize a large bus in the center of the frame, while treating fire indicators or a fallen person behind it as mere background noise.

This occurs because the model’s attention mechanism is spread across multiple targets, resulting in shallow contextual understanding of critical risk signals.

2. 💡 The Solution: Extending Cognition with Q₂E​

To address this issue, we adopted the core philosophy of Q₂E: “Complex events become easier to understand when they are decomposed.”

2.1. Key Insight from Q₂E (Query-to-Event)​

The Q₂E paper demonstrates that instead of searching with a single term like “wildfire”, decomposing it into an event flow—precursor → progression → outcome—allows the model to retrieve and understand the event far more effectively.

2.2. EVA Agent Adoption: Scenario Decomposition​

We extended this idea into Multi-Scenario Understanding. Rather than asking a VLM to observe everything broadly at once, we assign it distinct perspectives, each focused on a specific scenario.

• Before (Single Perspective): “Notify me if there is a fire, a fallen person, or a traffic accident.” → (Ambiguous, attention dispersed)

• After (Decomposed Perspectives):

  1. [Fire Perspective]: “Focus on pixel changes, colors, and smoke textures to identify signs of fire.”
  2. [Safety Monitoring Perspective]: “Analyze human pose and interactions with surrounding objects to detect falls.”
  3. [Traffic Monitoring Perspective]: “Examine vehicle collisions or abnormal stopping behavior to determine accidents.”

By decomposing the query this way, the VLM’s visual encoder activates strong, scenario-specific attention. Even when viewing the same image, the model gains clarity on what it should perceive, enabling it to detect previously overlooked situations.

3. 🏛️ System Design: A Specialized Multi-Agent Architecture​

Overall Pipeline​

The EVA Agent processes complex scenarios through the following five stages:

1. Classifier (Single or Multi): Determines the number of scenarios in the input
2. Scenario Decomposition: Splits multi-scenario inputs into individual scenarios
3. Scenario Enrichment: Injects rich detection conditions into each scenario [4]
4. Scenario Clustering: Groups similar scenarios when the count exceeds four
5. Parallel Inference: Executes VLM inference in parallel and aggregates results (Conceptual illustration: a single image processed through four separate prompt paths)

Stage 1: Scenario Classification​

The LLM analyzes the user-provided scenario description to determine whether it represents a single or multiple scenarios.

[e.g.]
"If a fire occurs, or someone falls, or a traffic accident is detected"

[LLM Prompt]
Analyze the following scenario description and determine whether it is
a single scenario or multiple scenarios:
"{user_input}"

[LLM Output]
{
  "type": "multi",
  "count": 3,
  "scenarios": ["fire", "fall", "traffic accident"]
}

Comparison with Q₂E:

• Q₂E: Performs event decomposition from a single query like “2025 LA Fire”
• EVA: Performs scenario decomposition from compound queries like “fire or fall or traffic accident”

Stage 2: Scenario Decomposition​

Just as Q₂E decomposes a complex query into Prequel / Current / Sequel, we decompose complex scenes into domain-specific directives.

[Q₂E Decomposition]
Query: "2025 LA Fire"
→ Prequel: "What could happen before?"
→ Current: "What happens during?"
→ Sequel: "What could be the outcome?"

[EVA Decomposition]
Query: "fire or fall or traffic accident"
→ Scenario 1: "Fire indicator detection"
→ Scenario 2: "Fall accident detection"
→ Scenario 3: "Traffic accident detection"

Stage 3: Scenario Enrichment​

Comparison with Q₂E Refinement:

Enrichment Example (See also: Turning Simple User Requests into AI-Understandable Instructions)

Stage 4: Scenario Clustering (When N > 4)​

When the number of enriched prompts exceeds four, semantically similar scenarios are grouped to improve inference efficiency.

Clustering Rule:

IF N > 4:
    Ask LLM to group scenarios by visual similarity
    into a maximum of four groups
ELSE:
    Process each scenario independently

[Example with many scenarios]
Input: ["fire", "smoke", "explosion", "fall", "collapsed person",
        "traffic accident", "collision", "emergency vehicle"]

[LLM-based Semantic Grouping]
Group 1 (Fire-related): ["fire", "smoke", "explosion"]
Group 2 (Human safety): ["fall", "collapsed person"]
Group 3 (Traffic): ["traffic accident", "collision"]
Group 4 (Emergency response): ["emergency vehicle"]

→ Reduce 8 scenarios to 4 groups
→ Optimize VLM latency and memory usage

Stage 5: Parallel VLM Inference​

Each enriched prompt is sent to the VLM in parallel.

[e.g.]
enriched_prompts = [Group1_prompt, Group2_prompt, Group3_prompt, Group4_prompt]

PARALLEL_EXECUTE:
    FOR each prompt IN enriched_prompts:
        result = VLM(frame=cctv_image, instruction=prompt)
        results.append(result)

WAIT_ALL_COMPLETE

Key Differences:

• Q₂E analyzes a single video across multiple modalities (video, audio, text)
• EVA analyzes a single frame across multiple scenarios (fire, fall, traffic)

4. 📊 Case Study: Unified vs. Decomposed Scenarios​

This is not about asking more questions—it is about how accurately the model understands the situation.

Using real EVA evaluation footage, we compared unified scenarios and decomposed multi-scenarios on the same complex video.

4.1. Test Scenario​

A video containing three critical incidents played sequentially was evaluated using both approaches.

Unified Scenario Prompt​

"If at least one emergency vehicle exists,
or a fire occurs,
or at least one person has fallen"

Decomposed Multi-Scenario Prompts​

Each scenario includes enriched detection criteria and exceptions.

4.2. Experimental Results​

4.3. Detailed Analysis​

1) Limitations of Unified Scenarios (Attention Interference)​

VLM Output

Analysis:
✅ One fallen person detected (Scene 1)
✅ One emergency vehicle (police car) detected (Scene 2)
❌ No fire indicators detected

Result: One critical incident missed

Issues:

• Attention bias toward early scenes (people, vehicles)
• Subtle fire indicators (smoke, flame textures) ignored in favor of salient objects

2) Decomposed Scenario Detection Result​

[Fire Detection Result]
⚠️ Fire detected

Detected visual features:
✓ Orange/red flames (Scene 3, bottom-right)
✓ Vertical gray smoke patterns
✓ Irregular bright region boundaries

Location: Bottom-right of frame
Confidence: 0.91
Decision: Fire confirmed (immediate dispatch required)

Why it worked:

• Specialization: Fire Agent focuses exclusively on fire-related features
• Interference Reduction: Other scenarios do not introduce perceptual noise

5. Conclusion: “The Depth of the Question Determines the Depth of Understanding”​

This work leads to a clear conclusion. What limits VLM performance is not the number of parameters, but how we instruct the model to perceive the world.

Just as Q₂E improves retrieval accuracy by decomposing queries, we demonstrated that Scenario Decomposition enables VLMs to simultaneously and deeply understand multiple situations in complex CCTV monitoring environments.

Moving forward, we plan to extend this approach beyond static image analysis toward Temporal Event Decomposition, enabling models to reason over the temporal context of video streams.

(This post presents a creative adaptation of the Q₂E: Query-to-Event Decomposition methodology to solve Multi-Scenario Cognition challenges in vision-based tasks.)

References​

[1] Shubhashis Roy Dipta, Francis Ferraro. "Q₂E: Query-to-Event Decomposition for Zero-Shot Multilingual Text-to-Video Retrieval." arXiv:2506.10202v2, 2025.

[2] Standley, T., Zamir, A., Chen, D., Guibas, L., Malik, J., & Savarese, S. "Which Tasks Should Be Learned Together in Multi-task Learning?" ICML 2020.

[3] Pratt, S., Covert, I., Liu, R., & Farhadi, A. "What Does CLIP Know About a Red Circle? Visual Prompt Engineering for VLMs." ICCV 2023.

[4] EVA Tech Blog: Turning Simple User Requests into AI-Understandable Instructions

Accidents in industrial environments happen without warning. Moments such as a worker collapsing, an arm getting caught in machinery, or a fire breaking out usually occur within seconds.

Physical AI should not stop at recognizing these moments. It must be capable of translating perception into physical action on site.

In this post, we walk through a LEGO-based simulation to show how EVA detects incidents and how its decisions are connected to real equipment actions as a single, continuous flow.

Simplifying Industrial Scenarios with LEGO​

Instead of replicating complex industrial environments in full detail, we simplified accident scenarios using LEGO.

We designed independent scenarios for:

• a worker collapsing,
• an arm being caught in equipment,
• and a fire breaking out.

EVA: Interpreting Situations as Events​

What matters in this simulation is not simply detecting a person or recognizing flames.

EVA interprets each situation through predefined detection scenarios and evaluates them as meaningful events.

Below is the interface where detection scenarios are configured in EVA.

Detection immediately becomes a trigger condition for deciding the next action.

Physical Action Trigger: When AI Decisions Move Reality​

When an event occurs, EVA’s role does not end at detection.

EVA transforms detected events into Physical Action Triggers, connecting them directly to on-site equipment and devices so they can respond immediately.

The key point is that each accident scenario is mapped to a predefined physical response. Without waiting for human intervention, AI decisions are translated directly into real-world actions.

Through this structure, AI judgments do not remain as on-screen alerts or logs. They become actions that actively change the state of the现场.

Physical Action Triggers represent the point where AI moves beyond “what it sees” to executing what must change in the real world.

EVA → n8n → Equipment Control Workflow​

These Physical Actions are not implemented with hardcoded logic. They are built using a workflow-based approach.

Detection events generated by EVA are delivered to n8n via Webhooks. Based on the severity and context of the event, an Agent within n8n sends the appropriate control signals to on-site equipment.

With this structure, even if equipment changes or scenarios expand, workflows can be reused and adapted flexibly.

A Structure That Makes Physical AI Tangible​

This LEGO simulation does not replicate a real industrial site in full detail.

However, the structure— where an incident occurs, AI perceives it, and a decision leads to a physical action— is identical to real-world environments.

EVA does not leave AI as a result on a screen. It enables AI to directly intervene in the physical world, realizing the concept of Physical AI.