Optical discs aren’t so immortal after all

Writeable CDs and DVDs, now being used millions of times a day to store everything from corporate tax returns to Little League pictures, may seem like one of the little wonders of the digital age. This little wonder, though, has a big problem: No one really knows how long the discs will last.

Fortunately, a mild-mannered civil servant at a small federal agency is stepping forward to meet the challenge. His name is Fred Byers, technical staff member at the U.S. National Institute of Standards and Technology.

Mr. Byers is the federal government’s go-to person on the care and handling of CDs and DVDs. Last year he wrote a comprehensive guide to the topic, an issue that is of no small importance to any number of federal agencies, starting with the Library of Congress and the IRS. Do a Web search on “Byers CD care guide,” and it comes up right away.

Currently it can be found HERE.

Also, check out these:
Something’s rotten in CD land: Optical discs aren’t so immortal after all

Breaking point

CD-R / DVD-R lifespan may be less than 2 years

This story appeared last year, but thought I’d reproduce it as it hasn’t been widely distributed, and is important.

Ever Decreasing Circles

You know those CD-Rs that you’ve trusted your most precious memories to?
They could be little more use than coasters after just two years.

Michael Pollitt, 21 April 2004

Are we putting too much faith in the ubiquitous “recordable CD”, or CD-R? It is undeniably one of the most useful means of storage around, offering an inexpensive way to save digital photographs, music and files and costing less than 50 pence per disc.

If you check the claims made by some manufacturers of popular CD-R brands, you will see that some make bold claims indeed. Typical boasts include: “100-years archival life”, “guaranteed archival lifespan of more than 100 years” and “one million read cycles”. One company even says data can be stored “swiftly and permanently”, leaving you free to bequeath those backups of your letter to the electricity company to your great-great-grandchildren.

But an investigation by a Dutch personal computer magazine, PC Active, has shown that some CD-Rs are unreadable in as little as two years, because the dyes in the CD’s recording layer fade. These dyes replace the aluminium “pits” of a music CD or CD-Rom, and the laser uses that layer to distinguish 0s from 1s. When the CD is written, the writing laser “burns” the dye, which becomes dark, to represent a “1” while a “0” will be left blank so that if the dye fades, there’s no difference; it’s just a long string of nothing to the playback laser.

So have you already lost those irreplaceable pictures you committed to the silver disc? PC Active suggests we should forget CD-Rs as a durable medium, after its own testing found some with unreadable data after just two years. “Though they looked fine from the outside, they turned out to be completely useless,” wrote the technical editor Jeroen Horlings, who had tested 30 brands in 2001, left them in a dark cupboard for two years and then re-tested them in August 2003. Of the brands tested, 10 per cent showed ageing problems. And it wasn’t just Horlings. After seeing the results, shocked readers contacted the magazine with their experiences.

Recordable DVDs are not off the hook either. The “dye chemicals” in write-once DVDs are similar to CD-R, though recording density and disk construction differ. “We’re in the process of testing DVDs and we’re sure that the same problems will occur,” said Horlings, who plans to publish his findings soon.

Gordon Stevenson, the managing director of Vogon International – a company specialising in data recovery – is familiar with these shortcomings thanks to the experiences of his customers, one of whom commissioned Vogon to retrieve pictures of his second honeymoon from a failed six-month-old CD-R. “The dye layer was fading,” Stevenson says, “but we were able to recover most of the disk. But these claims [of a 100-year archival life] are unhelpful and misleading. If you’re spending 20p on something, you probably don’t expect it to last 100 years,” he says.

In the wrong conditions, such as sunlight, humidity and upper surface damage, your CD-R will slowly turn into a coaster. “CD-Rs should never be left lying in sunlight as there’s an element of light sensitivity, certainly in the poor quality media,” says Stevenson. “I wouldn’t rely on CD-Rs for long-term storage unless you’re prepared to deal with them as recommended.”

Such views are echoed by the National Archives at Kew. “Generally speaking, we don’t recommend CD-Rs for long-term storage,” says Jeffrey Darlington, a project manager at the Archives’ Digital Preservation Department. “We don’t regard CD-Rs as an archival medium. Most of the CD-Rs on the market are not of archival quality.” Instead of CD-Rs, therefore, the National Archives tend to use magnetic tape rated for a 30-year life. Also, they are careful to copy, check and re-copy to avoid losing information and this is also a useful strategy for CD-Rs. “If you keep doing that so the CD-R is never more than physically three to five years old, you’ll be safe enough. A hundred years sounds pretty unlikely,” says Darlington.

Not all optical media is vulnerable. The rewritable variants (RW) use metallic materials that change the phase of the light, rather than light-sensitive dyes. Commercial magneto-optical and ultra-density optical systems are different too. Stewart Vane-Tempest, the optical product director at Plasmon, the archival specialists, has first-hand experience of unreadable CD-R media. “Some dyes are very robust, but others not,” Vane-Tempest says. “The one thing they have in common is susceptibility to environmental conditions. I do a lot of digital photography and pay top price for media. If I have anything important, I generally make a couple of copies. I’ve not used CD-Rs for long-term archiving.”

Vane-Tempest also offers a tip. Blank CD-R disks have a code that your CD writer reads to find the best writing strategy. If this isn’t in the CD-writer’s inbuilt software (its “firmware”), the default may be a poor compromise. Vane-Tempest says that some “less scrupulous” Far East companies have been using other people’s codes, with deficient results. However, there is a way around this which is to find out which brands suit your writer and ensure the firmware is up to date.

While such matchmaking is useful, there’s no way to assess CD-R longevity at home. All you can do is check periodically. As for whether manufacturers are guilty of using finger-in-the-air methods, Kevin Jefcoate, the marketing and product management director at Verbatim, says: “It’s a bit more than guesswork because there’s a lot of scientific evidence to back it up.”

The answer, Jefcoate says, is to use a climate chamber to accelerate the ageing of the organic dye. Using a relationship between chemical reaction rate and temperature, 100-year lifetimes may be argued for normal conditions. Jefcoate adds that he has never known users to complain of age-related failures? “We haven’t had anyone complain that, after three to five years, it hasn’t worked.” It’s easy to blame budget CD-Rs when things go wrong. Novatech’s purchasing and product manager, Kriss Pomroy, suggests users buy a small quantity for testing first.

The PC builder sells unbranded CD-Rs sourced from a Far East distributor that buys over-production from well-known factories. Are we saving pennies and taking risks? “No,” says Pomroy, “You can get problematic batches, but that’s as true with branded media.” The company now sells two-and-a-half times more unbranded write-once DVDs than CD-Rs.

The world’s No 1 supplier of CD-Rs, Imation, talks of “saving precious digital photo memories” – exactly what many people think they’re doing. Semar Majid, its technical marketing executive, hasn’t heard of any ageing problems. “Optical media should last between 30 and 200 years,” he says, “but it’s dependent on storage conditions and how you handle it.” He suggests transferring important photos to DVD, and keeping on moving to new formats.

Another big maker, TDK, takes a cautious view with DVDs, claiming only a 70-year lifespan. “This does not mean that DVD is more fragile or unstable in time compared to CD-R; this is only because of the shorter experience that we have in manufacturing and testing this relatively young technology,” says the TDK product manager Hartmut Kulessa. There have been no complaints about ageing failures.

As the oldest CD-R is barely a teenager, there are no definitive answers either. But perhaps the last word belongs to Jeroen Horlings at PC Active. “We see a lot of manufacturers and they think that quantity is more important than quality,” he says. “The problem will remain.”

For more info on CD-Rs and dyes:

The Differences Between Film & Digital Video

Found this article while searching for some informaton on video preservation. Bloke form Kodak has some intersting things to sat about Digital vs Analogue mediums. Remember that the film the shot motion pictures is the same 35mm format ands pretty much the same negative emulsions as the film shot in 35mm still cameras (“cameras”).

The Differences Between Film & Digital Video

The following is a conversation with Tom Wallis, chief technical officer, Kodak Entertainment Imaging Division, who explains how images are recorded on film and digital video media, how the processes are alike and how they are different, and the possibilities for the future.

QUESTION: What happens when light strikes a frame of motion picture film?

WALLIS: Every frame of film has a thin emulsion layer composed of randomly distributed microscopic silver halide crystals that are suspended in a three-dimensional pattern. When photons of light strike the emulsion, a photo-electrical reaction takes place on these microscopic “solid state sensors.” These sensors react logarithmically to light and form a latent image consisting of tiny silver atoms embedded within the crystal structure.

QUESTION: How does the latent image become a picture?

WALLIS: When a laboratory processes the negative, the latent image is chemically amplified. This amplification chemically produces the dye images that form the picture in a color negative film. Since the eye reacts logarithmically to light, the resulting dye image is very close to what you see through the viewfinder.

QUESTION: How much control does the cinematographer have?

WALLIS: They have enormous creative freedom to interpret images by their choice of negative, the way they expose the film, the absence or presence of light, the color of light, the use of filters and diffusion, and how the film is handled by the lab. Films can be “pushed” or “pulled” and there are various other proprietary lab processes. Cinematographers can also manipulate colors, contrast and sharpness during digital postproduction and they can alter colors optically during timing in the lab.

QUESTION: What are the differences between various camera films?

WALLIS: Every negative is designed to provide specific imaging characteristics, including speed or sensitivity to light, grain, contrast and colors.

QUESTION: What happens when light strikes a CCD sensor?

WALLIS: With a CCD array, you have a rigid two-dimensional grid of pixels (picture elements) on the sensor. As photons strike each pixel, a photo-electrical reaction takes place kicking off electrons in a linear, rather than logarithmic, fashion proportional to the amount of light received. These electrons are stored in a “well” for subsequent amplification, electrical processing and storage on magnetic media. This fixed grid format is a key difference between the imaging systems. The random nature of the silver halide grains and resulting differentiation frame to frame is why you do not see such artifacts as aliasing with film.

QUESTION: What are the other differences between film and digital video?

WALLIS: Another key difference is dynamic range or in other words the capacity for recording shades of tonality and colors from darkest shadow to the brightest highlight. Film has a very broad dynamic range of 1,000:1 compared with about 100:1 with digital video cameras. The HD 24P camera performs well in the shadow areas when used at its rated speed but it still lacks the nuances you get with the “film look”. At the other end of the scale, however, digital video will blow out or “clip” highlights in higher contrast scenes.

QUESTION: What about picture resolution?

WALLIS: When you scan film on state-of-the-art equipment, you can get 4,000 pixels of information across the horizontal axis of 35 mm color negative. This 4K image compares quite favorably to the 1920 pixels across the horizontal axis of the new 24P HD digital cameras. Super 16 mm color negative film has 2000 pixels of information across its horizontal axis.

QUESTION: Does that mean that Super 16 film and digital HD are equivalent?

WALLIS: No way! While resolution may be about the same in terms of pixel count, pictures are more than pixels. With film you have the dramatic dynamic range, the full gamut of color, the texture of the image and the complete “film look” which is not offered by the digital HD system.

QUESTION: Some video vendors claim Super 16 film is too grainy for HD display and they have displayed tests that seem to indicate that.

WALLIS: Kodak has done tests with well-exposed Super 16 film transferred on the Philips Spirit DataCine and subsequently broadcast through an HD delivery system. The results are stunning! My point is this: different tests by different testers will produce different results. For example, at a recent SMPTE conference, a video vendor compared a new HD scene to an older film scene using a stock that Kodak manufactured years ago. They also did not choose the best scanning technology to convert the film image to digital video format. Ultimately you convince yourself on the relative merits with your own tests done in a fashion that is meaningful for the project at hand.

QUESTION: Do you agree with claims that digital video needs less light for exposure?

WALLIS: We believe that the “fastest” digital video cameras have a relative exposure index of 400 to 500. The HD 24P cameras used with the new Panavision lenses seem to be more like 200 to 300 at the maximum. Our film speeds range from an exposure index of 50 to 800. You can “boost” video and gain speed by sacrificing other imaging characteristics. You can also “push” film and trade speed for grain. Here again, personal experience will go a long way to sort out the speed issues. It is very important not to fall into the trap of thinking that any of these technologies reduces the creative aspect of lighting. Images are made of light and the cinematographer uses light and shadows to create the image. The notion that you can save time by not lighting is ludicrous.

QUESTION: Is there a way to define the “film look”?

WALLIS: We try to define it but all we can do is describe it. The word that many people use is organic. I believe at least, in part, that’s because the random nature of the silver halide crystals is similar to the way our eyes work. Everyone has a certain number of rods and cones in their retinas but they aren’t on a fixed grid. They are random, so all of us see the same thing a little differently. Part of the “film look” is also how color is recorded. Our estimate is that our current negative films can resolve about 20 million different colors. Theoretically you can do this with digital video but today the technology has not advanced to that point either in gamut or performance in various light sources.

QUESTION: Can you predict when digital video might catch up with film?

WALLIS: I can envision building a digital imaging system that is comparable to today’s film system once we solve some technical problems and also figure out how to make it economically feasible. The problem is that digital video is aiming at a moving target. We are making improvements in the entire film system. We are designing new films with specialized imaging characteristics. We are also seeing improvements in lenses that are very important. If you can limit the scattering and flaring of light in lenses, you can make film more efficient. In addition, there will be new intermediate and print films that will retain more of the original image quality captured on the negative. Film labs are also making advances in processing and printing technologies and we are working to improve both optical and digital projection.

QUESTION: What about hybrid technologies?

WALLIS: During recent years Kodak has earned several ATAS awards and an AMPAS award recognizing our contributions toward advancing the convergence of film and digital technologies. We earned the Emmy Awards for developing the imaging system for the Philips Spirit DataCine and the AMPAS SciTech award for the Kodak Lightning Film Recorder. Our Cinesite subsidiary has also done some pioneering work with digital intermediate technology that has allowed cinematographers to create “looks” in a digital suite.

QUESTION: What about claims that film technology is mature?

WALLIS: Our scientists believe we have the capacity to improve film speed, or sensitivity to light, by a factor ranging from two to seven stops of efficiency. I’m not sure what we would do with a film that is seven stops faster. But the point is that we can make dramatic progress without making anyone’s investment in cameras or other hardware obsolete. Similar advances in video technology tend to make hardware obsolete.

QUESTION: Is speed the only possibility for progress in camera films?

WALLIS: By no means! You can apply advances in the efficient capture of light to improve other aspects of image structure, including grain and sharpness, or maybe you want a film that has a different spectral response to colors. Maybe we’ll design films with a tonal scale designed for scanning devices that are optimized for hybrid applications, such as television programming or commercials. The possibilities are almost limitless.

QUESTION: It has been said that HD images are inherently sharper than film.

WALLIS: That’s a complicated subject because a lot of things contribute to the perception of sharpness, including contrast and depth of field. One of the things that happens in electronic image capture is that you’re amplifying the edges of objects and people, which makes the images seem sharper. That’s actually a problem when a cinematographer wants to use selective focus to draw attention to something in a frame. As an alternative, if you want a totally sharp picture, you can begin with film and enhance the image during digital postproduction.

QUESTION: How is today’s digital imaging technology different from the old analog systems that have been around since 1956?

WALLIS: With the old video systems, when light hit a photo-multiplier tube it was converted into an analog signal which was amplified, processed and then stored on magnetic media. Today, you use a digital camera to record a scene in much the same way, except you have a more tightly spaced sensor grid instead of a photo-multiplier tube. That basically allows you to sample at a higher density. Digital cameras have also eliminated image-smearing problems and there are a lot more controls on the camera. Real progress has been made.

QUESTION: Some people have speculated that Kodak is scaling down its investment in film technology. Is that accurate?

WALLIS: I suspect that may be wishful thinking by our competitors. We are investing a very significant part of a very large research budget in advancing silver halide technology. We are also investing in modernizing and building high-tech film manufacturing plants. We believe that film and digital imaging will co-exist during the foreseeable future with each offering its own benefits.

QUESTION: What about archivability? Is that an issue?

WALLIS: That’s an important question, because most people produce films for posterity. They want to be able to re-release and re-purpose the original film. I’ll give you an example. DVDs are usually longer than the original films because you want to intercut out-takes that the audience hasn’t seen before. Today, film is, by far, the best solution for archiving. Black and white separations will last for up to 500 years and color negative and intermediate stocks will last for hundreds of years, presuming they are stored correctly. Digital video is an improvement over analog video for archiving purposes because it consists of ones and zeros that enable lossless duplication. However, both digital and analog video are stored on comparatively volatile magnetic tape and disk media, and once a digital signal is gone, it’s gone forever. The other problem is obsolescence of formats. Some 74 video formats have been introduced since 1956, and even if the media survived, in many or most cases, there is no equipment for playback.

QUESTION: Is there a solution?

WALLIS: You can transfer important video programs to film for archiving, however most people who do that don’t invest in converting out-takes that might be needed in the future. In the longer term, we envision a digital recording medium that is more archival than magnetic tape. We are doing the research but we are probably a few years away from delivering a commercial product and then we will need standards and hardware before it becomes practical.

Schapelle Corby Update

Various extras in the news last night and this morning regarding the Corby case.

Indonesian Justice?

For more than an hour, Corby’s lawyers Lily Lubis and Erwin Siregar read a 75-page document called “Does Justice Still Exist for a Defendant in This Beloved Country”.

They attacked the prosecution for neglecting important clues. “Worst of all,” lawyer Lily Sri Rahayu Lubis said, “they failed to seek truth and justice” and “manipulated facts” to make Corby look guilty.

They focused on breaches of law committed when customs officers detained Corby at Denpasar airport on October 8 after finding the large bag of marijuana in her bodyboard bag.

“The customs officer did not follow the standard procedure on searching,” they said.

The officers had failed to weigh the bags individually and collectively, making it impossible to compare the weights of the luggage at check-in in Australia, and to check Corby’s claim that someone had put the marijuana into her bag.

They criticised the customs officers for failing to fingerprint her bag and the two plastic bags inside it that contained the marijuana to see if Corby had handled it. They had refused requests by Corby’s sister, Mercedes, and defence lawyers to fingerprint the plastic bag which contained the marijuana to see if it held Corby’s prints. Customs officers had allowed the bag to be contaminated by their own fingerprints.

Of the four customs officers who detained Corby at the airport, three spoke no English and the one who spoke a little English acted as translator. Corby’s lawyers said all these officers relied on this one translator and so all had the same, wrong version of events.

Lawyer Erwin Siregar said the prosecution had accepted evidence by a customs officer who spoke “poor English” even though Corby, her 16-year-old brother, James, and friends disputed his version. The officer said Corby had refused to open the boogie board bag. Corby and her brother said she opened it without being asked.

Customs officers said Corby had been present when they took the contents – the boogie board, flippers and the marijuana – out of the boogie board bag and that Corby had identified herself as the owner of all the items.

But James testified that he had been ordered to take the boogie board to the customs interview room where he was ordered to remove the contents. Corby was not in the room at the time. When she came to the room, the items were laid out on a table. She had never admitted the marijuana was hers.

How “stupid would she be”, Mr Siregar asked, to pay $50,000 in Australia for marijuana that would sell in Bali for $10,000? How would Corby get the money to buy the marijuana in Australia?

The prosecution had rejected evidence by Corby, her mother, sister, brother and a friend that they had seen Corby put the flippers in the boogie board bag in Brisbane and there had not been marijuana in the bag.

It had rejected evidence by an Australian policeman that criminals in Australia transported drugs by placing them in luggage and removing them at the destination airport.

The prosecution had also rejected evidence by a witness who had overheard others describing how the marijuana was put into Corby’s boogie board bag.

“Please ignore the information provided by the five witnesses [all customs officers],” they said.

Under Indonesian law, the onus is on an accused person who claims innocence to say who is guilty of an offence.

Corby’s lawyers said it was “impossible” for them to do this in this case but they pressed their argument the drugs were placed in Corby’s bag at Brisbane Airport after she checked in.

“It’s a public secret that the domestic airports in Australia are used by drug syndicates to transfer or move drugs from one place to another by using the unlocked luggage of innocent passengers.”

Her lawyers described prosecutors as “street magicians” for claiming the marijuana was of high quality even though they had always refused their request to have it tested to find out its potency and where it was grown.

“Now the same prosecutors who rejected our demand to conduct tests on the marijuana have stated that marijuana is of very high quality,” they said.

Corby’s dad says no drugs in boogie board bag

“I fixed the board, there was a little strip missing off the boogie board thing, there was just a little plastic strip, so we stuck that on, stuck it in the bag,” Michael Corby told ABC’s 7.30 Report.

“I put it in the car, she put her bag in the car, kissed goodbye, [said] ‘Have a good time.’

“Her mother took her to the airport at 4.30 in the morning, got all the photos there, the smiling face, going goodbye.

“Brisbane to Sydney. Sydney to Bali. She opens her bag for them there, they didn’t ask her to, and all that marijuana stuff is in the bag.”
. . .
“She had nothing to do with bloody drugs,” he said.

“Oh, she might have had a puff when she was in bloody grade 10 or something, around the back of the schoolyard like kids do, I don’t know.

“She had nothing to do with it since, or any time as far as I know.

“She’s against it, anti-drugs.

“Anyway, I’d seen the bloody bag there, there was nothing in it.”. . .

Mr Corby, who has prostate cancer, said he felt helpless as his daughter fought to clear her name.

Indonesian Judge read book “Life Imprisonment” while Schapelle gave her appeal.

As she bit her lip and set her jaw to keep fraying emotions in check, it was lucky Schapelle Corby could not read Indonesian. Had she understood the language used to prosecute her, she might have noticed the title of the book one of her judges was reading, Life Imprisonment.

Outside the court, Judge I Gusti Lanang Dauh explained he was reading the book before deciding Corby’s sentence.

“Because there is a demand from the prosecutors for a life sentence, I am reading this book as a reference to add to my knowledge.”

It was still too early to reveal if he would give prosecutors what they had requested.

“That’s a secret,” he said.


Corby’s final plea to judges, The Age

I don’t know how long I can survive, SMH

Father bewildered over Corby’s plight, ABC

Defence accuses prosecutors of failure to seek justice

US pressuring Iraq to buy American instead of Australian?

It appears that the new Iraq administration is under a bit of pressure from the U.S. to buy wheat from them instead of Australia, who had been Iraq’s wheat supplier since the first gulf war.

While Iraq has paid Aussie for its recent wheat shipments, it has refused to accept them saying that they are contaminated with iron particles, though recent independant tests show that the wheat is top quality. Aussie what growers are hopping mad.

Cartoonist Leunig penned this amusing cartoon published in The Age:

Further Story Links:
Iraq Asks Australia AWB To Send Delegation On Wheat Issue
Iraq rejects Aussie wheat
Australia, U.S. scuffle over wheat to Iraq

All I did wrong was not locking my bag: Corby

All I did wrong was not locking my bag: Corby – World – www.smh.com.au:

“‘Firstly, I’d like to say to the prosecutors I cannot admit to a crime I did not commit,’ she told the Denpasar District Court.

‘And to the judges, my life at the moment is in your hands but I would prefer that my life was in your heart.’

‘And I say again that I have no knowledge of how the marijuana came to be inside my bag, and I believe the evidence shows (that), one, there is a problem in Australia with security at airports and baggage handling procedures.

‘Two, my own mistake is not putting a lock on on my luggage.
‘Three, I have never at any stage claimed ownership of the plastic bag and its contents.

‘Four, had the police weighed all of my luggage for the total weight, it would have proven to show a difference from the total weight at check-in at Brisbane airport.’

In a voice cracking with emotion, she told the judges she had already been punished enough for doing no more than failing to lock her bags.

“I believe the seven months I have been in prison is severe enough punishment for not putting locks on my bags,” she said tearfully.

Corby said her family’s reputation and her own had been “severely burdened” since her arrest last October at Bali’s Denpasar airport with 4.1kg of marijuana in her unlocked body board bag.

“I don’t know how long I can survive in here,” she said.

“I swear that as God is my witness, I did not know that the marijuana was in my bag.

“Please look to your God for guidance in your judgment for me. For God only speaks of justice, and your Honours, I ask for you to show good judgment and send me home.

“I am the innocent victim of a … drug smuggling network.

“I am not a person involved in drugs and I am not a person who might become involved in a drug smuggling operation.”

Members of the Corby family had earlier decorated the courtroom with symbolic yellow ribbons and had written placards demanding her freedom.

Onlookers in the packed court applauded as Corby completed her statement that she had penned in her prison cell.”