Editing Motion capture (section)

==Optical systems==
''Optical systems'' utilize data captured from image sensors to [[triangulation (computer vision)|triangulate]] the 3D position of a subject between two or more cameras calibrated to provide overlapping projections. Data acquisition is traditionally implemented using special markers attached to an actor; however, more recent systems are able to generate accurate data by tracking surface features identified dynamically for each particular subject. Tracking a large number of performers or expanding the capture area is accomplished by the addition of more cameras. These systems produce data with three degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of three or more markers; for instance shoulder, elbow and wrist markers providing the angle of the elbow. Newer hybrid systems are combining inertial sensors with optical sensors to reduce occlusion, increase the number of users and improve the ability to track without having to manually clean up data.<ref>{{Cite book |last1=Li |first1=Jian |last2=Yang |first2=Jiushan |last3=Xu |first3=Zhanwang |last4=Peng |first4=Jingliang |title=2012 International Conference on Image Analysis and Signal Processing |chapter=Computer-assisted hand rehabilitation assessment using an optical motion capture system |date=November 2012 |url=http://dx.doi.org/10.1109/iasp.2012.6425069 |pages=1–5 |publisher=IEEE |doi=10.1109/iasp.2012.6425069|isbn=978-1-4673-2546-2 }}</ref>

===Passive markers===
[[Image:MotionCapture.jpg|thumb|A dancer wearing a suit used in an optical motion capture system]]
[[File:Motion capture facial.jpg|thumb|Markers at specific points on an actor's face during facial optical motion capture]]
''Passive optical'' systems use markers coated with a [[retroreflective]] material to reflect light that is generated near the camera's lens. The camera's threshold can be adjusted so only the bright reflective markers will be sampled, ignoring skin and fabric.

The centroid of the marker is estimated as a position within the two-dimensional image that is captured. The grayscale value of each pixel can be used to provide sub-pixel accuracy by finding the centroid of the [[Gaussian]].

An object with markers attached at known positions is used to calibrate the cameras and obtain their positions, and the lens distortion of each camera is measured. If two calibrated cameras see a marker, a three-dimensional fix can be obtained. Typically a system will consist of around 2 to 48 cameras. Systems of over three hundred cameras exist to try to reduce marker swap. Extra cameras are required for full coverage around the capture subject and multiple subjects.

Vendors have constraint software to reduce the problem of marker swapping since all passive markers appear identical. Unlike active marker systems and magnetic systems, passive systems do not require the user to wear wires or electronic equipment.<ref>{{cite journal|title=Motion Capture: Optical Systems|journal=[[Next Generation (magazine)|Next Generation]]|issue=10|publisher=[[Imagine Media]]|date=October 1995|page=53}}</ref> Instead, hundreds of rubber balls are attached with reflective tape, which needs to be replaced periodically. The markers are usually attached directly to the skin (as in biomechanics), or they are [[velcro]]ed to a performer wearing a full-body spandex/lycra [[Mo-cap suit|suit designed specifically for motion capture]]. This type of system can capture large numbers of markers at frame rates usually around 120 to 160 fps although by lowering the resolution and tracking a smaller region of interest they can track as high as 10,000 fps.

===Active marker===
[[File:Body Motion Capture.jpg|thumb|Body motion capture]]
Active optical systems triangulate positions by illuminating one LED at a time very quickly or multiple LEDs with software to identify them by their relative positions, somewhat akin to celestial navigation. Rather than reflecting light back that is generated externally, the markers themselves are powered to emit their own light. Since the inverse square law provides one quarter of the power at two times the distance, this can increase the distances and volume for capture. This also enables a high signal-to-noise ratio, resulting in very low marker jitter and a resulting high measurement resolution (often down to 0.1&nbsp;mm within the calibrated volume).

The TV series ''[[Stargate SG1]]'' produced episodes using an active optical system for the VFX allowing the actor to walk around props that would make motion capture difficult for other non-active optical systems.{{citation needed|date=August 2016}}

ILM used active markers in ''[[Van Helsing (film)|Van Helsing]]'' to allow capture of Dracula's flying brides on very large sets similar to Weta's use of active markers in ''[[Rise of the Planet of the Apes]]''. The power to each marker can be provided sequentially in phase with the capture system providing a unique identification of each marker for a given capture frame at a cost to the resultant frame rate. The ability to identify each marker in this manner is useful in real-time applications. The alternative method of identifying markers is to do it algorithmically requiring extra processing of the data.

There are also possibilities to find the position by using colored LED markers. In these systems, each color is assigned to a specific point of the body.

One of the earliest active marker systems in the 1980s was a hybrid passive-active mocap system with rotating mirrors and colored glass reflective markers and which used masked linear array detectors.

===Time modulated active marker===
[[Image:Activemarker2.PNG|thumb|300px|A high-resolution uniquely identified active marker system with 3,600 × 3,600 resolution at 960 hertz providing real time submillimeter positions]]
Active marker systems can further be refined by strobing one marker on at a time, or tracking multiple markers over time and modulating the amplitude or pulse width to provide marker ID. 12-megapixel spatial resolution modulated systems show more subtle movements than 4-megapixel optical systems by having both higher spatial and temporal resolution. Directors can see the actor's performance in real-time, and watch the results on the motion capture-driven CG character. The unique marker IDs reduce the turnaround, by eliminating marker swapping and providing much cleaner data than other technologies. LEDs with onboard processing and radio synchronization allow motion capture outdoors in direct sunlight while capturing at 120 to 960 frames per second due to a high-speed electronic shutter. Computer processing of modulated IDs allows less hand cleanup or filtered results for lower operational costs. This higher accuracy and resolution requires more processing than passive technologies, but the additional processing is done at the camera to improve resolution via subpixel or centroid processing, providing both high resolution and high speed. These motion capture systems typically cost $20,000 for an eight-camera, 12-megapixel spatial resolution 120-hertz system with one actor.

[[Image:PrakashOutdoorMotionCapture.jpg|thumb|300px| [[Infrared|IR]] sensors can compute their location when lit by mobile multi-LED emitters, e.g. in a moving car. With Id per marker, these sensor tags can be worn under clothing and tracked at 500 Hz in broad daylight.]]

===Semi-passive imperceptible marker===
One can reverse the traditional approach based on high-speed cameras. Systems such as [http://web.media.mit.edu/~raskar/LumiNetra/ Prakash] use inexpensive multi-LED high-speed projectors. The specially built multi-LED IR projectors optically encode the space. Instead of retro-reflective or active light emitting diode (LED) markers, the system uses photosensitive marker tags to decode the optical signals. By attaching tags with photo sensors to scene points, the tags can compute not only their own locations of each point, but also their own orientation, incident illumination, and reflectance.

These tracking tags work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. The system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Since the system eliminates a high-speed camera and the corresponding high-speed image stream, it requires significantly lower data bandwidth. The tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique appears ideal for on-set motion capture or real-time broadcasting of virtual sets but has yet to be proven.

===Underwater motion capture system===
Motion capture technology has been available for researchers and scientists for a few decades, which has given new insight into many fields.

====Underwater cameras====
The vital part of the system, the underwater camera, has a waterproof housing. The housing has a finish that withstands corrosion and chlorine which makes it perfect for use in basins and swimming pools. There are two types of cameras. Industrial high-speed cameras can also be used as infrared cameras. Infrared underwater cameras come with a cyan light strobe instead of the typical IR light for minimum fall-off underwater and high-speed cameras with an LED light or with the option of using image processing. [[File:Oqus underwater.jpg|thumb|Underwater motion capture camera]]
[[File:Motion tacking by using image processing.PNG|thumb|Motion tracking in swimming by using image processing]]

=====Measurement volume=====
An underwater camera is typically able to measure 15–20 meters depending on the water quality, the camera and the type of marker used. Unsurprisingly, the best range is achieved when the water is clear, and like always, the measurement volume is also dependent on the number of cameras. A range of underwater markers are available for different circumstances.

=====Tailored=====
Different pools require different mountings and fixtures. Therefore, all underwater motion capture systems are uniquely tailored to suit each specific pool instalment. For cameras placed in the center of the pool, specially designed tripods, using suction cups, are provided.

===Markerless===
Emerging techniques and research in [[computer vision]] are leading to the rapid development of the markerless approach to motion capture. Markerless systems such as those developed at [[Stanford University]], the [[University of Maryland]], [[MIT]], and the [[Max Planck Institute]], do not require subjects to wear special equipment for tracking. Special computer algorithms are designed to allow the system to analyze multiple streams of optical input and identify human forms, breaking them down into constituent parts for tracking. [[ESC entertainment]], a subsidiary of [[Warner Brothers Pictures]] created especially to enable [[virtual cinematography]], used a technique called Universal Capture that utilized [[multi-camera setup|7 camera setup]] and the tracking the [[optical flow]] of all [[pixel]]s over all the 2-D planes of the cameras for motion, [[gesture]] and [[facial expression]] capture leading to photorealistic results.

====Traditional systems====
Traditionally markerless optical motion tracking is used to keep track of various objects, including airplanes, launch vehicles, missiles and satellites. Many such optical motion tracking applications occur outdoors, requiring differing lens and camera configurations. High-resolution images of the target being tracked can thereby provide more information than just motion data. The image obtained from NASA's long-range tracking system on the space shuttle Challenger's fatal launch provided crucial evidence about the cause of the accident. Optical tracking systems are also used to identify known spacecraft and space debris despite the fact that it has a disadvantage compared to radar in that the objects must be reflecting or emitting sufficient light.<ref>{{Cite journal| doi = 10.1007/BF00216781| title = Optical tracking of artificial satellites| year = 1963| last1 = Veis | first1 = G.| journal = Space Science Reviews| volume = 2| issue = 2| pages = 250–296| bibcode=1963SSRv....2..250V| s2cid = 121533715}}</ref>

An optical tracking system typically consists of three subsystems: the optical imaging system, the mechanical tracking platform and the tracking computer.

The optical imaging system is responsible for converting the light from the target area into a digital image that the tracking computer can process. Depending on the design of the optical tracking system, the optical imaging system can vary from as simple as a standard digital camera to as specialized as an astronomical telescope on the top of a mountain. The specification of the optical imaging system determines the upper limit of the effective range of the tracking system.

The mechanical tracking platform holds the optical imaging system and is responsible for manipulating the optical imaging system in such a way that it always points to the target being tracked. The dynamics of the mechanical tracking platform combined with the optical imaging system determines the tracking system's ability to keep the lock on a target that changes speed rapidly.

The tracking computer is responsible for capturing the images from the optical imaging system, analyzing the image to extract the target position and controlling the mechanical tracking platform to follow the target. There are several challenges. First, the tracking computer has to be able to capture the image at a relatively high frame rate. This posts a requirement on the bandwidth of the image-capturing hardware. The second challenge is that the image processing software has to be able to extract the target image from its background and calculate its position. Several textbook image-processing algorithms are designed for this task. This problem can be simplified if the tracking system can expect certain characteristics that is common in all the targets it will track. The next problem down the line is controlling the tracking platform to follow the target. This is a typical control system design problem rather than a challenge, which involves modeling the system dynamics and designing [[motion controller|controllers]] to control it. This will however become a challenge if the tracking platform the system has to work with is not designed for real-time.

The software that runs such systems is also customized for the corresponding hardware components. One example of such software is OpticTracker, which controls computerized telescopes to track moving objects at great distances, such as planes and satellites. Another option is the software SimiShape, which can also be used hybrid in combination with markers.

====RGB-D cameras====
RGB-D cameras such as [[Kinect]] capture both the color and depth images. By fusing the two images, 3D colored [[voxels]] can be captured, allowing motion capture of 3D human motion and human surface in real-time.

Because of the use of a single-view camera, motions captured are usually noisy. Machine learning techniques have been proposed to automatically reconstruct such noisy motions into higher quality ones, using methods such as [[lazy learning]]<ref>{{cite journal |last1=Shum |first1=Hubert P. H. |last2=Ho |first2=Edmond S. L. |last3=Jiang |first3=Yang |last4=Takagi |first4=Shu |title=Real-Time Posture Reconstruction for Microsoft Kinect |journal=IEEE Transactions on Cybernetics |date=2013 |volume=43 |issue=5 |pages=1357–1369 |doi=10.1109/TCYB.2013.2275945|pmid=23981562 |s2cid=14124193 }}</ref> and [[Gaussian]] models.<ref>{{cite journal |last1=Liu |first1=Zhiguang |last2=Zhou |first2=Liuyang |last3=Leung |first3=Howard |last4=Shum |first4=Hubert P. H. |title=Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models |journal=IEEE Transactions on Visualization and Computer Graphics |date=2016 |volume=22 |issue=11 |pages=2437–2450 |doi=10.1109/TVCG.2015.2510000|pmid=26701789 |s2cid=216076607 |url=http://nrl.northumbria.ac.uk/id/eprint/25559/1/07360215.pdf }}</ref> Such method generates accurate enough motion for serious applications like ergonomic assessment.<ref>{{cite journal |last1=Plantard |first1=Pierre |last2=Shum |first2=Hubert P. H. |last3=Pierres |first3=Anne-Sophie Le |last4=Multon |first4=Franck |title=Validation of an Ergonomic Assessment Method using Kinect Data in Real Workplace Conditions |journal=Applied Ergonomics |date=2017 |volume=65 |pages=562–569 |doi=10.1016/j.apergo.2016.10.015|pmid=27823772 |s2cid=13658487 |doi-access=free }}</ref>