KNOW HOW Backups
Have You Really Backed up?
Tape problems
When you desperately need to restore data will it be there? Is your software
the right choice for such an important task.
BY ADRIAN KERTON
our hard drive has crashed, but backup hardware will be dictated by the Within a tape drive, the data presented
there s an inner contentment. Not amount of data you want to backup, but to the tape is often manipulated in struc-
Yonly did you back up the whole don t be fooled by what at first sight tured ways to ensure that it gets the best
system last night, but you keep a spare appears to be obvious. distribution of flux transitions on to the
new hard disk drive at hand for just such tape. This makes sure your data is in the
Case 1
an eventuality. most robust format there can be. It is not
No Problem; just put in the new disk, You re safe. Your drive has Read After unknown for something to go wrong
partition and format it, add the operating Write (RAW). between the data connector and the
system, install the backup programme, Just about all tape drives have read write head. In such cases Read After
(or on Unix and Linux use tar or cpio), after write capability. This means that Write will report all is well because the
and watch the tape drive recover the there is a read head positioned just after read head reads the data that the write
whole lot whilst you have some coffee. the write head and the tape drive verifies head wrote, but this data is not the data
You ll have it all restored before most that what it reads is exactly what it has you wanted to write! It is corrupt, and if
people get in for work. written. If there is drop out on the tape something has gone badly wrong it could
You start the restore, but why is the Read After Write will detect it, the be random data.
tape drive thrashing around so much? backup application will try to write the The result? Garbage on the restore.
Your heart sinks as you realise the data again and if there are problems
Golden Rule
restore is not restoring. An error message it will move down the tape and write
appears on the screen. It doesn t matter again on a good piece of the tape, so no Don t rely on RAW or backup software
what the message says, you know you problem there! Or is there? (including Unix/Linux embedded apps)
are now in deep trouble. The restore has Some backup applications rely on the that relies on Read After Write. ALWAYS
failed and you know you cannot get at read after write function within the tape run a verify pass on your backup. If the
your information! drive to serve as the backup verification backup application does not support
You eject the tape and put in the pre- mechanism, but there are a lot of hurdles verify, ditch it for one that does and do it
vious backup tape, OK it s last week s, in the way of the data trying to get to the now before it is too late.
but that s better than nothing. Misery tape head.
Case 2
sets in as the same thing happens again. If the data is going across a network
The phone rings. Your boss and his boss then the problems are magnified as cor- You go to restore a data file and find it is
want to know why their computers are ruption can occur anywhere in the not on the backup set.
not on line. You start to explain and they network hardware or software before it Why? Because the backup application
make it quite clear that if you cannot get gets close to the backup device. Read had a complicated user interface and
their computers on-line in an hour, you After Write won t help you if you present you misunderstood the include/exclude
won t have a job at the end of that hour. corrupt data to the tape drive. feature on the backup application, or
Why? What could have brought you to Consider data coming off a disk and you mistyped the latest free backup
this miserable episode? going to a SCSI tape drive on the same command line programme by one letter.
Well there are a number of causes and computer. It travels from the disk, onto Result you only backed up system files
it s probably worth examining them to the bus to memory and then back from when in fact you wanted to include only
make sure you don t get caught like this. memory to the bus to the SCSI host bus data files.
Analysts have determined that 20-30% adapter (HBA) where the software driver This can easily happen when a new
of backups fail, and the user doesn t has to be correctly matched to the oper- job is created, because once a backup
even know it. ating system. Then through the adapter job has been created it does its job each
It doesn t matter which technology hardware to the SCSI cable to the tape time running in the background, and the
you are using to backup; tape, disk, drive, where the tape drive s firmware administrator forgets all about it. When
optical, whatever, there are some golden needs to match the adapter card driver. a new job is required the administrator
rules you need to follow. Generally the Finally through the tape drive hardware. has to  relearn the application because
54 April 2003 www.linux-magazine.com
Backups KNOW HOW
it is used so rarely, sometime only once approach is if one block of data cannot is bad, the entire backup will be lost
every two or three years. Often, during be read during a recovery, data from the even though segments of the backup set
the needs for a new backup job the multiple clients will be lost. Also restores proved to be accurate.
administrator has changed, so the are complex requiring the management It should be noted that a tar archive
person creating the new backup job has of the multiple tape sets just to recover a can be fully verified using a bit by bit
to start from scratch with a package they single client system. check against the disk. This doubles the
have never seen before and no one else time it takes to do a complete backup.
Golden Rule
is around to act as a mentor. Nothing must change on the disk
Choose a backup application that has between the backup and verify, other-
Golden Rule
built in error handling. Surprisingly very wise errors will be generated and each
ALWAYS try a restore from a backup few backup applications can satisfacto- error will have to be investigated. This
whenever a new backup job has been rily accommodate errors during a approach is impractical because of
configured (to a test directory is useful) restore. Check with the software com- today s shrinking backup windows.
to make sure the files you want are pany to understand what they do to Some applications note the problems
actually there. You should always do this ensure the availability of the data. with a backup and record them in the
even if you have run a verify pass, as this An application s bells and whistles are fault log. It is very easy to forget to check
will only verify that the files you selected no good if the underlying technology the log, particularly if it is someway
to be backed up are actually there. If you cannot deliver the data. Your data is down the directory chain, so you will not
selected the wrong files, verify alone will important, so meticulous care should be know if your backup has failed.
not help you. taken to check a backup software s capa-
Golden Rule
It helps if your backup package is easy bilities to fully understand the level of
to use and doesn t have too many bells protection it affords. Make sure your backup application
and whistles to learn. Don t choose a incorporates the checks and balances
Case 4
backup package that does everything, to assure that the data you believe
unless you really need the extras. You backed up with a verify pass, you backed-up actually made it to the
the restore runs perfectly, but then the backup media accurately and can be
Case 3
complaints start rolling in. The data successfully and accurately recovered.
You have backed up, run a verify pass has errors, some files are in error with Without this assurance, all other appli-
and a restore, but 3 months later the characters missing. cation functionality is window dressing.
restore fails with some error message, Why? When a backup application is Make sure your backup application has
that usually says the restore will be based on the cpio format, the checksums some sort of notification that alerts you
aborted. Typically tar or cpio will gener- used to verify the data s accuracy are when there is a problem with the
ate  tape I/O read error and the restore only calculated on the meta data (data backup, usually by email.
will be aborted.  about the data block), and does not
Finally
Why? The backup application met a checksum the actual data. Therefore a
bad spot on tape and quite rightly found cpio verify pass cannot verify the actual  I don t need backup  I ve got RAID.
an error because it couldn t read the data data is correct, only that the header RAID is fault tolerant, it is not fault
properly. Now you have the first few files information is correct. free. The Internet is full of tales of RAID
from the backup, the bulk of it is still on Some backup applications verify the arrays that fail. Remember also that
the tape. This is typical of a backup backup by conducting test restores on users deleting their data is one of the
application that is just a user interface random backup sets. The same issue most common causes of lost data, and in
built on top of tar or cpio. previously addressed applies here. If the that case RAID will not help you. If the
Another problem that might arise is backup data hasn t been 100% verified, building catches fire, the RAID array may
when the backup application uses multi- users can still experience aborted be lost, but a tape backup made with a
streaming from different client systems restores because corrupt data can still be reliable backup package and stored off-
to the same tape. In this technique, data experienced. If the first bit of the restore site will save the day.
from one client group is Backing-up data is a sim-
interleaved with data from ple concept; just move data
other client groups onto the to a safe place and bring it
same tape. This means that back when it s needed. In
any particular client group s reality, how this work gets
data will be divided and done is very complex. The
spread amongst the data of process should not be an
the other client groups on  art form, but good science
the backup media. If the and engineering.
backup is large it may have The availability of your
spread over a number of data, and your sanity,
tapes. The danger in this Figure 1: Read After Write Logic on the tape depends on it. %
www.linux-magazine.com April 2003 55