the new math: old battle of the lovees was: PDP1 3572

It is the resolution. Generally the the accuracy is specified with an in some regions along the scale, excluding cumulative error. I've cross-checked it with other calipers and there is agreement to 4 places to the extent I've checked. I had a really cheapo dial caliper that had .005" cumulative error in about 1" of travel which I soon discarded.

My cheapo dial calipers are not specified to conform to any standards, but you can buy good ones that are.

If the the resolution of the instrument is .00001, and a range of 3 inches, you can measure from 0.00001 through 2.99999, for example. So if I had such a nice instrument, I could measure my nominally 1.5" steel shaft and find out it is in fact perhaps 1.48932" in diameter at one end, and chart how the diameter varies over its length, in increments of .00001. At each point I would have 6 significant places since with the nice instrument, I could tell the difference between 1.48932" and 1.48933". With my cheapo calipers I couldn't tell any difference.

I have a Starret inside micrometer with a set of shafts that stack up providing inch increments from 1" to about 12", with a micrometer that resolves within 1" in .001" steps. I used it to establish the X and Y dimensions down to thousanths of the ellipse formed by the mouth of a 6.5" diameter steel pipe that I threaded and turned, to make an extension for my dad's telescope mounting pier. If I was measuring inside a 10" bore, my inside mic would be measuring 5 significant places because I could tell the difference between 10.001" and 10.002". Inside mics are tricky since its extremely easy to misposition them and get a false reading.

For accuracy, you have to express the magnitude and type of errors. The dimensions specified in chip manufacturing will probably include error tolerances as well.

Gregm