Strange bad sequential write speed of the H240ar disk controller in RAID1 mode

Hi all!

I am currently doing setup of server DL360 Gen9 (K8N32A) with H240ar controller (2GB cache + battery) and 2 SAS HDD (15k).
I have updated BIOS and the latest Firmware (v3.56 for H240ar).


I had the strange results when done dumb testing with sequential writes using "dd" (OS FreeBSD 10.2).

1) When I did the test in HBA mode (software mirror) I got excellent results (I compared the results with two other similar servers):

# diskinfo -t da0
da0
        512             # sectorsize
        300000000000    # mediasize in bytes (279G)
        585937500       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        71806           # Cylinders according to firmware.
        255             # Heads according to firmware.
        32              # Sectors according to firmware.
                0TV423MJ        # Disk ident.

Seek times:
        Full stroke:      250 iter in   1.963112 sec =    7.852 msec
        Half stroke:      250 iter in   1.407693 sec =    5.631 msec
        Quarter stroke:   500 iter in   1.495174 sec =    2.990 msec
        Short forward:    400 iter in   0.410938 sec =    1.027 msec
        Short backward:   400 iter in   1.137867 sec =    2.845 msec
        Seq outer:       2048 iter in   0.116620 sec =    0.057 msec
        Seq inner:       2048 iter in   0.269316 sec =    0.132 msec
Transfer rates:
        outside:       102400 kbytes in   0.408835 sec =   250468 kbytes/sec
        middle:        102400 kbytes in   0.457162 sec =   223991 kbytes/sec
        inside:        102400 kbytes in   0.562953 sec =   181898 kbytes/sec

-----------------------------
# dd if=/dev/zero of=/jails/tmp.dat bs=2048k count=2k
2048+0 records in
2048+0 records out
4294967296 bytes transferred in 1.551414 secs (2768421105 bytes/sec)

-----------------------------
# dd if=/dev/zero of=/jails/tmp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 35.009251 secs (3067023135 bytes/sec)



2) But then I decided to use the Hardware RAID1 mode with r/w cache (default settings). And I got the odd bad result:

# diskinfo -t da0
da0
        512             # sectorsize
        299966445568    # mediasize in bytes (279G)
        585871964       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        71798           # Cylinders according to firmware.
        255             # Heads according to firmware.
        32              # Sectors according to firmware.
        PDNLH0BRH9E8OX          # Disk ident.

Seek times:
        Full stroke:      250 iter in   0.400514 sec =    1.602 msec
        Half stroke:      250 iter in   0.755768 sec =    3.023 msec
        Quarter stroke:   500 iter in   2.386514 sec =    4.773 msec
        Short forward:    400 iter in   1.228658 sec =    3.072 msec
        Short backward:   400 iter in   0.980554 sec =    2.451 msec
        Seq outer:       2048 iter in   0.154139 sec =    0.075 msec
        Seq inner:       2048 iter in   0.188522 sec =    0.092 msec
Transfer rates:
        outside:       102400 kbytes in   0.187210 sec =   546979 kbytes/sec
        middle:        102400 kbytes in   0.299430 sec =   341983 kbytes/sec
        inside:        102400 kbytes in   0.371288 sec =   275797 kbytes/sec

-----------------------------

# dd if=/dev/zero of=/jails/tmp.dat bs=2048k count=2k
2048+0 records in
2048+0 records out
4294967296 bytes transferred in 16.863843 secs (254684967 bytes/sec)



3) I tried to disable write cache, but it didn't help:

# diskinfo -t da0
da0
        512             # sectorsize
        299966445568    # mediasize in bytes (279G)
        585871964       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        71798           # Cylinders according to firmware.
        255             # Heads according to firmware.
        32              # Sectors according to firmware.
        PDNLH0BRH9E8OX          # Disk ident.

Seek times:
        Full stroke:      250 iter in   0.713020 sec =    2.852 msec
        Half stroke:      250 iter in   0.727876 sec =    2.912 msec
        Quarter stroke:   500 iter in   2.306020 sec =    4.612 msec
        Short forward:    400 iter in   0.761613 sec =    1.904 msec
        Short backward:   400 iter in   0.944677 sec =    2.362 msec
        Seq outer:       2048 iter in   0.125513 sec =    0.061 msec
        Seq inner:       2048 iter in   0.120609 sec =    0.059 msec
Transfer rates:
        outside:       102400 kbytes in   0.418003 sec =   244974 kbytes/sec
        middle:        102400 kbytes in   0.459038 sec =   223075 kbytes/sec
        inside:        102400 kbytes in   0.565851 sec =   180966 kbytes/sec

-----------------------------
# dd if=/dev/zero of=/jails/tmp.dat bs=2048k count=2k
2048+0 records in
2048+0 records out
4294967296 bytes transferred in 17.832703 secs (240847799 bytes/sec)

-----------------------------
# dd if=/dev/zero of=/jails/tmp.dat bs=2048k count=50k
51200+0 records in
51200+0 records out
107374182400 bytes transferred in 511.838806 secs (209781246 bytes/sec)


I see that according to the test results the "Transfer rates" parameter better when the cache active, but this is not consistent with the results of tests with "dd".
And I'm very surprised that the results when you disable write cache for RAID1 does not coincide with the results for HBA-mode.

Can anyone explain this strange result?
This result is a consequence of incorrect system settings or a feature of the disk controller?
For example, do I need to align the disks when using RAID mode of H240ar disk controller (this was done in HBA mode)?