Accuracy Performance of the Close-Range Photogrammetry System iWitness: Results of a Measurement Comparison with a Surveying Total Station - May, 2007


Preamble

The iWitness photogrammetry software system, Version 1.3, combined with a consumer grade digital camera, (a 10MP SONY DSLR-A100) was assessed for accuracy potential against an electronic distance measuring device (a total station.) As described in this report, the results indicated an accuracy agreement to an average of .036’ RMS (or .432”) over a test range area spanning approximately 75 feet (wide) by 125 feet (long).


Background

This validation test of 3D measurement accuracy was carried out in conjunction with members of the North Carolina State Highway Patrol, Asheville, Concord, Nags Head, and Oak Island Police Departments during the DeChant Consulting Services – DCS Inc., iWitness photogrammetry workshop on May 23, 2007, sponsored by the North Carolina Department of Justice Academy NCDOJ, in Edneyville, NC. The aim of the validation test was to compare the iWitness photogrammetric measurement approach utilizing a consumer grade digital camera to yield a similar level of accuracy to that of the total station measuring device.


The validation test site was set up at the NCDOJ test-track. A screen capture of one of the images used in the iWitness software for 3D measurement is noted below.


Accuracy study of iWitness at NCDOJ overall shot

iWitness user interface showing referenced image with measured points.

The validation test comprised of not only comparing the total station to the iWitness image-based measurement method, but also to validate the use of the DCS photogrammetric markers www.photomeasure.com/em.html (being iWitness semi-automatic centroid dot measured, relative to the marker’s white dot being .25’ elevated off the ground.) The target “offsetting” capabilities of iWitness compared to the measurements made with the total station prism rod (i.e., tip of prism rod placed on the ground.)


The overall measurement process comprised of three stages: Firstly, the 3D coordinates of total station were shot by two police officers. Secondly, the image-based measurements were made using the police officers Sony digital camera (photogrammetrically calibrated by iWitness) and DCS photogrammetric markers. Thirdly, the results of both measurement technologies were imported into a CAD system; after a least squares coordinate transformation was accomplished within iWitness, whereas both data sets were in a common coordinate system for validation-test comparison.


Scene Recording by Total Station

The total station was set up where two police officers worked measuring the locations of “blue-chalk cross-marks” assigned to the asphalt, previously made by the test team.  As each measurement was made, the total station’s prism rod tip left a ‘circular impression’ in the blue chalk, approximately 1mm in diameter. 


iWitness points compared to total station

DCS photogrammetric markers aligned over blue caulk marks for the test.


The DCS representative followed along after every total station prism rod measurement was made and carefully placed the photogrammetric marker centroid dot, over the total station rod tip impression, so the marker dot alignment was in fact collinear with the rod tip impression. With the photogrammetric marker placed over the prism rod tip impression (based strictly on a visual basis of marker placement) the aim was to project the photogrammetric marker centroid dot .25’ in the –Z direction, (note: the +/- Z-axis direction being the photogrammetric coordinate systems up/down direction.)


The total station operators measured a 100’ distance with the total station, as verified by a brand new steel tape.  The same measurement ( the 100’) was measured with the total station after the validation test was accomplished.  More discussion on the 100’ steel tape test measurement will be described later on in the reporting.


It took approximately 45 minutes for the total station to be set up and record for the full acquisition of the 3D data points. It should be noted, this validation test was not aimed at one method being “faster than the other”; it was simply to try and see how well the two metrology methods agreed to each other using a consumer grade digital camera, and a typical commercially sold total station used by local and state police departments.


Scene Recording by Photogrammetry (digital camera and photogrammetric markers)

With the photogrammetric markers placed over the prism rod measured locations on the asphalt, the only thing remaining to do was measure at least one photogrammetric scale distance. By using the same steel tape as the total station, a quantity of two vector measurements were made being 30’, and 7’ for photogrammetric scale.  Again, photogrammetric markers were placed over the two scale distances marked on the asphalt.


For the test, twenty-three digital images were used from good perspective angles; however, all images were taken by one person from the same elevation off the ground, which is quite typical for quickly ‘3D mapping’ a scene during the needs of accident reconstruction. 


It took approximately 20 minutes to set the photogrammetric scale distances, along with accomplishing the picture taking, (where 39 images were actually acquired, and only 23 were measured in iWitness for the validation test.) The iWitness measurements, were made on a NCDOJ Academy PC, by an officer of the Concord, NC Police Department, during iWitness training with DCS Inc., the week of May 21st, 2007.


Comparison of 3D Measurement Results

The actual coordinate comparison, after least squares coordinate transformation of the two data sets, was then reviewed within a CAD diagramming program. In a 'best fit' of one 3D data set to the other, coordinate discrepancies were better than 1 inch, with the root mean square error being .036'. This approximate 7/16” level of comparison represents the measurement accuracy of the two 3D measurement methods, but more of a comparison within a routine AR environment.


As documented elsewhere, iWitness is capable of performing 3D measurements to better than 1:50,000 of the size of the object given the right combination of camera, targets and strong network geometry.


Review of the steel tape measurements Total Station Vs Photogrammetry


As mentioned in this reporting, the steel tape was used for at least three measurements, (1) by the total station (100’ distance) and (2) for the photogrammetry, (30’ and 7’).  Using the total station and measuring the 100’ distance after the validation test, revealed it was 99.97 feet (or .03’ difference with the steel tape being the standard.)   Measuring the photogrammetry distances within iWitness, revealed 30.0002’ and 6.9994’, respectively.   The summary of the iWitness internal accuracy estimates is noted below.


Internal accuracy report of iWitness


Internal accuracy estimates of iWitness, revealed the RMS 1-sigma level average of .004’; which equates to approximately 1mm in the overall distance of the validation test, or the overall accuracy of 1:22500 of object measurement size and validated by steel tape 'check distances'.  iWitness agreed to the steel tape to approximately 1mm, while iWitness agreed to the total station within .036’ RMS.  It is clear the accuracy of the total station in this validation test setup, was not as accurate as the photogrammetry approach, however for “AR purposes”, the accuracy of both systems is clearly comparable for the level of 3D accuracy needed in forensic accident reconstruction.


The following screen capture is the iWitness and the total stations 3D data points in the same coordinate system, after doing a least squares coordinate transformation with iWitness (the functionality within iWitness is called “3D transformation to control”.) 


The data sets were imported as a DXF file from iWitness, and also an ASCII text file was created from the total station for use in the Crash Zone diagramming program.  The only slight “inconvenience” in comparing the data sets was flipping the total stations Northing & Easting coordinates (i.e., LH coordinate system) to Easting & Northing (i.e., RH coordinate system) so that both total station and photogrammetry data sets were in the Right Hand Coordinate Reference Frame.  The following view is from a top-looking-down perspective.  The measured points of both total station and photogrammetry are just about a perfect overlay in respect to the accuracy requirements necessary for accident reconstruction.


Plan View of scene mapped with iWitness and total station

Both 3D data sets as viewed in CAD.


The below screen capture is an expanded view of the above…

Expanded view of iWitness Vs total station


Finally, noted below, a close up of marked “S” for the photogrammetry point, and the total station point…

Fractions of an inch comparison

Testing the iWitness Natural Target Marking Vs the Total Station

To further warrant the accuracy of iWitness as compared to a total station, three “yellow caulk cross-marks” were made, shot by the total station, and also “natural feature marked” in iWitness (see below screen capture).


3 point comparison iWitness Vs total station


The accuracy of these three iWitness marked points (named Road1, Road2 and Road3) agreed to the total station to an RMS fit of .067', .094', and .038’ (respectively). See below 3D transformation results in the iWitness dialog.


3D transformation to control dialog in iWitness


Summary

The impressive accuracy capability of iWitness in AR surveys was confirmed in the comparison of measurement results between the photogrammetric and total station approaches.  


Moreover, using a moderately priced digital camera with iWitness, police agencies across the USA, for more than five years now, have demonstrated the iWitness photogrammetry method provides a very fast 3D measurement tool for the police investigator; the acquiring of digital images at the scene for subsequent 'image-based mapping' takes only a matter of minutes. iWitness has been specifically designed for crime scene and accident scene mapping purposes, though it can also be applied in many other fields for 3D measurement. The system is very easy to use by non-specialists and it is by far the most robust close-range photogrammetry software available for applications within accident reconstruction and forensic measurement.


Acknowledgements

A number of individuals contributed to this validation test and the author appreciates their support.  A special thanks to the North Carolina Department of Justice, who were instrumental in spearheading the purchase of the iWitness close range photogrammetric technology for the aforementioned North Carolina Police Agencies.


 

Back to top