Login    Register

[EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Posts only related to Beta Builds, all others will be removed!
Forum rules
Set-Up GuideFAQsForum Rules
  • Author
    Message

[EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby RedAntz » 01 Aug 2013 10:36

This build includes experimental features and is meant for testing purposes. Do NOT use in production system!

New Experimental features
====================
1. Add loader.conf tab to 'System | Advanced'. This will allow users to adjust kernel variables, unload all modules, load modules. (*this only applies to embedded / full install)
2. Add driver for ARECA SATA/SAS HBA Controller ARC1320, ARC1300 . Note: This is not loaded by default. In embedded/full install, add arcsas_load="YES" via System|Advanced|loader.conf to load on boot.
3. Updates and improves hard disks detection, extends support for hard disks behind RAID controllers, and fixes S.M.A.R.T. issues not working in some setup.
4. Allows hard disks behind RAID controllers to display S.M.A.R.T. information (serial number, hard disk model, temperature etc. when supported)
5. Update the following pages :-
a) Diagnostics|Information|Disks page
- add columns : Description, Rotation rate, Transfer rate, S.M.A.R.T. status, Controller, Controller model
- rename column name 'Disk' to 'Device'
b) Disks|Management page
- add columns : Controller, Controller model
- rename column name 'Disk' to 'Device'
c) Status|Disks page
- add column : Device model
- rename column name 'Disk' to 'Device'

Download Location
==============
Only 64-bit builds are available for testing at this point.
https://www.dropbox.com/sh/8h4resx849zlzu4/jyNTqdt7W9

r833 : changes since 9.1.0.1.820.
- Add tw_cli (CLI for AMCC/3ware ATA RAID Controllers) . To 3ware cards owners: please provide console output of tw_cli so that further support for 3ware RAID controllers can be developed. Please contact me for further information.
- Update arcsas binary to V1.00.00.05
- Skip SCSI enclosure (ses) detection
- Fix and update HighPoint RocketRAID controllers detection (thanks dnar for reporting and testing !)
- Use default reported SATA transfer rate if current transfer rate is not provided in S.M.A.R.T.
- Show 'n/a' in Diagnostics|Information|Disks -> Transfer rate column if no value is available.
- Remove 'vfs.zfs.prefetch_disable="1"' from loader.conf as default settings. NAS4Free users can set this manually via "loader.conf" page.
- Fix "loader.conf" page not deleting last variable (thanks SIFTU for reporting and testing !)
- Delete loader.conf.local file if it is empty


Instructions
=========
Before you test this image, make sure you backup your config. (config.xml)

1. System|Backup/Restore: Backup your config from and restore your config.xml backup to this experimental build.
2. Disks|Management : after you restored your config form Step 1, you *MUST* click on "Clear config and Import disks" to update configuration.
3. Disks|Management|Disk|Edit: After step 2, you will need to re-activate S.M.A.R.T. monitoring for every device.

Note : If you have RAID controllers but cannot parse S.M.A.R.T. info properly, please add correct variables in System | Advanced | loader.conf (see loader.conf screenshot below) to load the correct kernel modules for controller support.Then reboot and "Clear config and Import disks" again to update configuration.

Have fun testing !

Testing
=====
a) Tested on the following platforms/controllers:
- HP Smart Array controller
- LSI 2008 (IT mode) on IBM ServeRAID M1015
- Intel ICH10 I/O Controller Hub
- Intel BD3420 PCH

b) * Supported but not tested (require further feedback from users):
- Areca SATA[/SAS] RAID controller
- HighPoint RocketRAID SATA RAID controller
- HighPoint RocketRAID 27xx controller

*Please note that this is still deemed *EXPERIMENTAL* as I do not have all RAID controller cards above to test. I have been contacting owners with 3ware, Areca and HighPoint RAID controllers for debug information to create this patch, but further development on these controllers are dependent on users' feedback on the patch. There are also chances where controllers/controllers description may be incorrect, depending on chipset and platform which I cannot thoroughly test due to lack of hardware at hand for thorough testing.

Please report in this thread for hard disks and IDE/SATA/SAS/RAID controllers that works/not working with this patch. Please do not hesitate to contact me at RedAntz at <NAS4Free domain> if you spot any hard disk/controller/S.M.A.R.T. related issues as well.

Future planning/additional controller support
==================================
3ware controllers :- tw_cli - CLI Storage Management Software for AMCC/3ware ATA RAID Controllers, supported by twe and twa drivers are added to experimental build 833. Output from cli console is required from 3ware users for further development. Please contact me if you can help.

For additional hardware support that is supported in FreeBSD 9.1, if you can help testing, let me know.

Screenshots
=========
System|Advanced|loader.conf
Image

console screenshot to show variables saved in loader.conf.local and kernel modules loaded
Image

Disk|Management screen after 'Clear config and Import Disks'
Image

Diagnostics|Information|Disks with IBM ServeRAID M1015 passthrough on ESXi
Image

Diagnostics|Information|Disks with HP Smart Array P410, IBM ServeRAID M1015 on bare metal booting off USB flash drive
Image
You do not have the required permissions to view the files attached to this post.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby jandegr » 11 Aug 2013 07:04

Hi,
The loader.conf tab is a nice feature, working for me on an embedded install :P
thx.

regards,
Jan
PowerUser
jandegr
 
Posts: 255
Joined: 23 Jun 2012 15:41
Location: Belgium

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby dnar » 11 Aug 2013 09:46

It sure is! No more need to edit and reboot following an upgrade. As it should be.

For those reading, the Highpoint 27xx cards are now tested and supported.
NAS 1: NAS4Free 9.1.0.1 x64 embedded, Intel Core 2 E8400, 8GB, 6 x WD Green 1TB in RAIDZ2, Highpoint 2720 SAS x 8 card 6Gbps
NAS 2: NAS4Free 9.1.0.1 x64 embedded, Intel Core i3-3220, 8GB, 7 x WD Green 750GB in RAIDZ1, onboard SATA 3Gbps

FOR SALE: 6 x 1TB WD Green drives. Lots of bad blocks, 29,000 hours on the clock, guaranteed for 1 minute or 1 mile which ever comes first...
PowerUser
dnar

User avatar
 
Posts: 113
Joined: 27 Apr 2013 08:38
Location: Perth Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby tuaris » 12 Aug 2013 02:28

The ARC1320 card seems to be working with this build. No issues so far.
PowerUser
tuaris

User avatar
 
Posts: 53
Joined: 19 Jul 2012 19:31

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby RedAntz » 12 Aug 2013 04:09

Thank you for your feedback guys !

This helps a lot to know what's tested and what works (or doesn't).

In the mean time. a new build (r826) is cooking ...
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby alexey123 » 12 Aug 2013 06:49

+1
The loader.conf tab is a nice feature, RedAntz, pls, upload his php code here.
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby RedAntz » 12 Aug 2013 09:17

alexey123 wrote:+1
The loader.conf tab is a nice feature, RedAntz, pls, upload his php code here.


Hi alexey123,

You can see grab codes committed to Sourceforge SVN http://sourceforge.net/p/nas4free/code/log/?path=. Relevant commits are r814 and r820.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby alexey123 » 12 Aug 2013 11:19

Many thanks/
I rebuild your loaderconfig as gui extension.

Image
.
I put all my extensions into /mnt/disk/app folder.
.
How to use it. Connect with NAS4Free as root
Code: Select all
cd /mnt/disk/app
fetch https://sites.google.com/site/aganimkarmiel/home/free-time/file-sharing/loaderconfig.tar.gz
tar xvf loaderconfig.tar.gz
loaderconf/loaderconf_install.php


If loaderconfig extension need after reboot on embedded , simple add Postinit script
Image
You do not have the required permissions to view the files attached to this post.
Last edited by alexey123 on 13 Aug 2013 18:15, edited 1 time in total.
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby ChriZathens » 13 Aug 2013 17:03

Thanks a lot for this!
I will update my backup nas (HP Proliant Microserver N40L) from 804 embedded to 820 embedded. I suppose there won't be any issues, but I will report if there are any.

UPDATE: One bug I notice:
In Diagnostics| information no rotate rate is shown, but the device model is identified for my disks :
Capture.PNG

While is status|disks it is not...
Capture1.PNG

Everything else seems OK...

UPDATE 2:
Now that I was able to see the transfer rate of the disks I noticed that the 5th one was at 150MB/s. I had by mistake changed the bios settings last time (I use the hacked bios) and that change had created a bottleneck in disk performance. Now that I saw it in n4f I remembered that I had made that change and went back to bios to correct this mistake. Thanks a lot for this!!! :D

UPDATE 3: Disks|Management has also info missing:
Capture3.PNG

Besides device model, columns controller and controller model are empty. But in Diagnostics| information the relevant info is there:
Capture4.PNG



Bottom line, only Diagnostics| information retrieves all the info (except rotation speed in my case - BTW my hdds are @7200k)
You do not have the required permissions to view the files attached to this post.
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: MSI GF615M-P33
  3. CPU: AMD Athlon II X240
  4. RAM: 8GB DDR3
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 4x1.5TB and 4x2TB HDDs in RaidZ1 arrays ~ 9.1 TB usable disk space :)
  7. O/S: NAS4Free 9.1.0.1.847 -amd64 embedded
  8. Extra H/W: LSI 3081E-R SAS Controller
Backup Nas: HP N40L (5x500GB RaidZ - 8GB ECC RAM)
Forum Moderator
ChriZathens

User avatar
 
Posts: 577
Joined: 23 Jun 2012 07:14
Location: Athens - Greece

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby RedAntz » 14 Aug 2013 11:17

alexey123 wrote:Many thanks/
I rebuild your loaderconfig as gui extension.


Thanks ! I'm surprised by the warm receptions of this feature :o

However, please note that this is still an experimental feature, so there is a good chance that there will be bug(s) (in fact I am working on a bug fix now)

Features in experimental will eventually be incorporated to NAS4Free mainline distribution when deemed stable.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby RedAntz » 14 Aug 2013 11:19

ChriZathens wrote:UPDATE: One bug I notice:
In Diagnostics| information no rotate rate is shown, but the device model is identified for my disks :
Capture.PNG

While is status|disks it is not...
Capture1.PNG

Everything else seems OK...


Thanks for embracing the experimental build to help testing !

Did you click on "Clear config and Import disks" to update configuration ?
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.820 - controller patch

Postby ChriZathens » 14 Aug 2013 12:35

No, actually I did not. I thought that it was not going to solve anything, since the device model is shown in a different page.
I will do it tomorrow and report my results! ;)
My Nas
  1. Case: Fractal Design Define R2
  2. M/B: MSI GF615M-P33
  3. CPU: AMD Athlon II X240
  4. RAM: 8GB DDR3
  5. PSU: Chieftec 850w 80+ modular
  6. Storage: 4x1.5TB and 4x2TB HDDs in RaidZ1 arrays ~ 9.1 TB usable disk space :)
  7. O/S: NAS4Free 9.1.0.1.847 -amd64 embedded
  8. Extra H/W: LSI 3081E-R SAS Controller
Backup Nas: HP N40L (5x500GB RaidZ - 8GB ECC RAM)
Forum Moderator
ChriZathens

User avatar
 
Posts: 577
Joined: 23 Jun 2012 07:14
Location: Athens - Greece

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby RedAntz » 15 Aug 2013 10:00

Hi all,

r833 experimental build is released. Please see first post for details. Thank you everyone for putting the effort to test and report this !

Hi alexey123,
I see that zoon01 is keen to incorporate loader.conf page to main trunk. So it is very possible that the new official release will have this feature sooner than I expected.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby alexey123 » 15 Aug 2013 10:14

RedAntz wrote:Hi alexey123,
I see that zoon01 is keen to incorporate loader.conf page to main trunk. So it is very possible that the new official release will have this feature sooner than I expected.


I'm happy, just need remove spaces into <variable><space>=<space><value>
Your code
Code: Select all
$lines .= $param['name'].' = "'.trim($param['value'])."\"\n";

Better
Code: Select all
$lines .= $param['name'].'="'.trim($param['value'])."\"\n";


I make this extension very quick, only copy-paste method was used. :D and for study install.php :mrgreen:
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby RedAntz » 15 Aug 2013 10:24

alexey123 wrote:I'm happy, just need remove spaces into <variable><space>=<space><value>

I make this extension very quick, only copy-paste method was used. :D and for study install.php :mrgreen:


Cool :) . You may want to include commits from r832 as well for the latest changes to loader.conf functionality.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby alexey123 » 15 Aug 2013 10:45

RedAntz wrote:Cool :) . You may want to include commits from r832 as well for the latest changes to loader.conf functionality.

I think in official release no need add file loader.config.local, need edit loader.conf.
But now I want work with my LPT project, I want add old upses (without driver ) for my NAS4Free servers, and add option to hard reboot windows server, when it freeze.
Image
:oops:

Also I have problem with CF card, connected to IDE slot. I wrote post about problem
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby RedAntz » 15 Aug 2013 11:08

alexey123 wrote:Also I have problem with CF card, connected to IDE slot. I wrote post about problem


I do not know how to limit disk transfer speed ( if I read your post correctly).
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby alexey123 » 15 Aug 2013 11:24

RedAntz wrote:I do not know how to limit disk transfer speed ( if I read your post correctly).

You read correctly. I search now to repair it, but I cannot find. Currently. sometimes I have message such in this post, from new CF card.
Sometimes = one week without messages, yesterday I receive more then 100 times, today without mesages :evil:
Code: Select all
kernel: (ada0:ata<bla>): WRITE_DMA. ACB: ca 00 7a 02 40 40 00 00 00 00 02 00
kernel: (ada0:<bla>): CAM status: Command timeout
kernel: (adaO:<bla>): Retrying command

Such messages I received from old 2.5 inch disks
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby jandegr » 15 Aug 2013 13:42

concerning loader conf.tab

Hi, here some testing of loader.conf tab due to upgrade from 820 to 833.

In version 820 : I made several entries in loader.conf tab and they were working fine.
Today I upgraded to version 833. Then entries in the loader.conf tab were still there,
but no loader.conf.local was created on /cf/boot so none of the entries were working.
Solution : edit one entry, no need to change anything, click on save and then on the
appy changes button. Now loader.conf.local is created on /cf/boot and after a reboot
all entries work again.

Maybe It would have worked right away if I did another reboot after the system came up the first time
after the upgrade, but I doubt it.

A nice feature and hopefully the loader.conf tab makes it to the mainstream release builds.

regards,
Jan
PowerUser
jandegr
 
Posts: 255
Joined: 23 Jun 2012 15:41
Location: Belgium

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby RedAntz » 15 Aug 2013 14:05

jandegr wrote:Maybe It would have worked right away if I did another reboot after the system came up the first time
after the upgrade, but I doubt it.


Hi Jan,

I've tested the normal backup config file and restore and it works.

I may have missed that upgrade path. I will investigate further tomorrow.
Developer
RedAntz

User avatar
 
Posts: 207
Joined: 11 Jul 2012 05:46
Location: Sydney, Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby siftu » 15 Aug 2013 15:10

alexey123 wrote:I think in official release no need add file loader.config.local, need edit loader.conf.


Loader.conf.local is something FreeBSD supports natively http://www.unix.com/man-page/freebsd/5/loader.conf/ . It seems a good solution since we currently generate loader.conf when building the images.
System specs: NAS4Free amd64-embedded on ASUSTeK. M5A78L-M LX PLUS - AMD Phenom(tm) II X3 720 Processor - 8GB ECC Ram, Storage: 2x ZFS mirrors with 4x Western Digital Green (WDC WD10EADS)
My NAS4Free related blog - http://n4f.siftusystems.com/
Moderator
siftu

User avatar
 
Posts: 118
Joined: 17 Oct 2012 04:36

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby alexey123 » 15 Aug 2013 15:57

siftu wrote:
Loader.conf.local is something FreeBSD supports natively http://www.unix.com/man-page/freebsd/5/loader.conf/ . It seems a good solution since we currently generate loader.conf when building the images.

I know about loader. Some as rc.conf.local, we use it for TheBrig.
But I think: use file loader.conf.local as additional file for experimental building or extension is TRUE way.
In official build must be only loader.conf, for give use loader.conf.local to developers.
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby alexey123 » 16 Aug 2013 07:22

I find bug.
If system partition destroyed, when I reinstall system loader.config.local need be recreated over webgui with such procedure:
Edit any variable, in my case I edited hw.ata.ata_dma=0 to hw.ata.ata_dma=1, NOT REBOOT server than revert old value hw.ata.ata_dma=0
Home9.2.0.1 - Sandstorm (revision 972) / x64-embedded on SAPPHIRE Pure Mini E350 / 4G RAM
Moderator
alexey123

User avatar
 
Posts: 1320
Joined: 19 Aug 2012 06:22
Location: Israel, Karmiel

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby dnar » 16 Aug 2013 11:45

I found that rsync server did not work with 820. Was there a reason for this? I am reluctant to try 833 unless I know what to expect with rsync, I rely on it daily.

Thanks for the update.
NAS 1: NAS4Free 9.1.0.1 x64 embedded, Intel Core 2 E8400, 8GB, 6 x WD Green 1TB in RAIDZ2, Highpoint 2720 SAS x 8 card 6Gbps
NAS 2: NAS4Free 9.1.0.1 x64 embedded, Intel Core i3-3220, 8GB, 7 x WD Green 750GB in RAIDZ1, onboard SATA 3Gbps

FOR SALE: 6 x 1TB WD Green drives. Lots of bad blocks, 29,000 hours on the clock, guaranteed for 1 minute or 1 mile which ever comes first...
PowerUser
dnar

User avatar
 
Posts: 113
Joined: 27 Apr 2013 08:38
Location: Perth Australia

Re: [EXPERIMENTAL] NAS4Free 9.1.0.1.833 - controller patch

Postby jandegr » 16 Aug 2013 15:47

dnar wrote:I found that rsync server did not work with 820. Was there a reason for this? I am reluctant to try 833 unless I know what to expect with rsync, I rely on it daily.

Thanks for the update.

Hi, I tested the following :

box 1 with rev 833 as rsync server
box 2 with rev 820 as rsync client

rsync worked exactly as before (rsync of a disk from the 833-server to the 820-client),
so no panic on my side, hope you can figure out what went wrong on your box.
:arrow: the rsync log's on both server and client should point you in the direction of the cause.

I also did a local rsync, from internal HD to an external USB HD, on the 833-box and it worked fine too.

regards,
Jan
PowerUser
jandegr
 
Posts: 255
Joined: 23 Jun 2012 15:41
Location: Belgium

Next

Return to Beta Builds (Testing Only)

Who is online

Users browsing this forum: No registered users and 2 guests