[
If you would like to get an
overview of the system, please download the main video first (by clicking here).]|
SmartSurv3D - A Distributed 3D Smart Surveillance System |
Sven Fleck, Florian Busch, Wolfgang Straßer
University of Tübingen, WSI/GRIS
[
If you would like to get an
overview of the system, please download the main video first (by clicking here).]The demand for surveillance systems has increased extremely over recent times. We present a system consisting of a distributed network of cameras that allows for tracking and handover of multiple persons in real time. The inter-camera tracking results are embedded as live textures in a 3D model which is available ubiquitously and can be viewed from arbitrary perspectives independent of the persons’ movements. We also briefly describe our way of hassle free 3D model acquisition to cover the complete system from setup to operation and finally show some results of both an indoor and an outdoor system in operation.
The top level architecture of our distributed surveillance and visualization system is given in Fig. 1. It consists of multiple, networking enabled camera nodes, a server node and a 3D visualization node. Each camera node is implemented by a smart camera. However, also a combination of one or multiple non-smart cameras in combination with a PC can be used within deprecated systems to allow for maximal flexibility.
Fig. 1 3D Surveillance System Architecture
Fig.
2 depicts the smart camera nodes where the distributed tracking is
implemented. Low bandwidth usage is achieved by embedding the
multi-person tracking process on smart cameras, whereas the results are
sent to a server which forwards the fused data to the visualization node.
Since nearly all the work is done on the camera, the system scales
ideally. The objects’ 2d pixel coordinates are converted into 3d space
by assuming a fixed ground plane. An adaptive background model enables
the automatic initialization of new target objects each tracked by a
color-based particle filter (Fig. 2 right).
Particle filters have become an attractive way of tracking, as they are
capable of handling multiple hypotheses (multimodal pdfs) and nonlinear
systems.
Fig. 2 Left. A smart camera node of our system. Right: Particle filter iteration loop.
The tracking algorithm is based on a color based particle filter. More details can be found here:
A
Smart Camera Approach to Real-Time Tracking
The server node manages configuration and initialization of all camera
nodes, collects the resulting tracking data and takes care of person
handover. Fig. 3 illustrates a screenshot of the server node.
Fig. 3 Server Node's Camera GUI showing the live view of a camera, overlayed by the actual tracking results.
A
practical 3D surveillance system also comprises an easy way of acquiring
3D models of the respective environment.
The basis for 3D model acquisition is our mobile platform which we call
the Wägele. It allows for an easy acquisition of indoor and outdoor
scenes: 3D models are acquired just by moving the platform through the
scene to be captured. Thereby, geometry is acquired continuously and
color images are taken in regular intervals. Our platform (see Fig. 4)
comprises an 8 MegaPixel omnidirectional camera C1 in conjunction with
three laser scanners L1-L3 and an attitude heading sensor A1.
Fig. 4
The Wägele. Two setups of our mobile platform. Left: Two Laser scanners
L1, L2 and one omnidirectional camera. Center &
Right: Three laser scanners L1, L2, L3 and omnidirectional camera
closely mounted together.
Two flows are implemented to yield 3D models: a computer vision flow based on graph cut stereo and a laser scanner based modeling flow. See here for futher details:
Wägele Platform for 3D Model Acquisition
Fig. 5 illustrates the indoor setup within our institute.
Fig. 5 Indoor Setup.
Indoor Results are shown in Fig. 6.
Fig. 6 Indoor Results. Renderings of the XRT Visualization Node. Left column: Output of the server node (Camera GUI): raw image of the camera node, overlayed with the target object on which a particle filter is running. (1),(4): Smart Camera, (7): webcam. Center Column: Rendering of embedded live texture in XRT visualization system. Right Column: Same as center, but with alpha map enabled: only segmented areas are overlayed for increased realism.
Results of an outdoor setup are shown in Fig. 7.
Fig. 7 Outdoor setup.
Outdoor Results are shown in Fig. 8 and Fig. 9.
Fig. 8 Outdoor experiment. (1) Background model. (2) Estimated noise. (3) Live smart camera view with overlayed tracking information. (4) Live XRT rendering of tracked object embedded in 3D model.
Fig. 8 Outdoor setup (1,8): Renderings of the acquired model in XRT visualization system. (2): Dewarped example of an omnidirectional image of the model acquisition platform. (3), (6) Live view of camera nodes with overlayed targets currently tracking. (4,5) Rendering of resulting person of (3) in XRT visualization system from two viewpoints. (9-11) More live renderings in XRT.
Overview video.
Accepted for Video Proceedings @ IEEE CVPR 2006
You can download additional videos here.
Links to other projects at WSI/GRIS:
A
Smart Camera Approach to Real-Time Tracking
Wägele Platform for 3D Model Acquisition
Omnidirectional 3D Modeling using Graph Cuts
Wall-based 3D modeling on a mobile robot
