Disk to Disk Copy strangeness

Hi Folks,

I'm currently copying data from one 90G raid 5 storage to an EVA system. The old system has a 90g partition that's quickily filling to capacity. So I created a 500G partition on the EVA.

I connected the drives, and can see them, etc. Now, I did a default format on the new 500G partition, and began copying over my message store (LOTS of small files).

Now the copy is executing, and the destination drive has exceeded the source data size by 40G.

When I look at the mkfs -m of each disk, I get the following:

SOURCE:

mkfs -F ufs -o nsect=64,ntrack=64,bsize=8192,fragsize=1024,cgsize=26,free=1,rps=90,nbpi=8268,o pt=t,apc=0,gap=0,nrpos=8,maxcontig=64

DESTINATION:

mkfs -F ufs -o nsect=128,ntrack=128,bsize=8192,fragsize=1024,cgsize=6,free=1,rps=60,nbpi=8244, opt=t,apc=0,gap=0,nrpos=8,maxcontig=128

I'm suspecting that something with my block size is causing the increase in disk usage, but don't know for sure. I'm hoping I can get some advice on how to make best use of the new 500g drive.

The data for the drive is a message store so a lot of small 1K-250K files.

-Tom

[1164 byte] By [TomR@UAAa] at [2007-11-15]
# 1
What program are you using to copy the data over? The copy program may be trying to map blocks as opposed to data.
domburnsa at 2007-7-12 > top of java,Storage Forums,Storage General Discussion...
# 2
The open source tool 'rsync'.-Tom
TomR@UAAa at 2007-7-12 > top of java,Storage Forums,Storage General Discussion...
# 3

Dunno enough about that tool to answer.

People used to have issues using dd, where they'd copy the entire disk (if=/dev/rdsk/cXtYDZs2 of=/dev/rdsk/cYtZdXs2) which might include the label, and any bad block info. If they used it between two disks of different sizes funny things could happen.

I'm thinking that if rsync uses a block level copy that the new unit is trying translate and thus expanding into the space that may be whats happening.

HTH

Dom

domburnsa at 2007-7-12 > top of java,Storage Forums,Storage General Discussion...
# 4
Thanks Dom!It's my understanding that rsync does file level copying since it copies files on a change basis (it copies deltas on subsequent rsyncs), but I'm confirming it now with the rsync community.-Tom
TomR@UAAa at 2007-7-12 > top of java,Storage Forums,Storage General Discussion...
# 5

Problem resolved.

The SunONE message store uses hardlinks to conserve disk space when mass emails are sent out. Single reference of the message is kept, and then hard links are established to each recipient mailbox.

I was not using the hard link option within rsync, so each hard link reference was being copied as a full file. Which accounts for the growth in disk usage.

-Tom

TomR@UAAa at 2007-7-12 > top of java,Storage Forums,Storage General Discussion...