Dear fellow Debian users,
[tl;dr] Please advice me on how to automatically figure out which disk (actually RAID array) is which during the early_command stage of an automated install while the RAID controller presents the disks to the kernel in random order and I want to use specific arrays for specific logical volumes.
At this moment, my question is: How can I make the arrays visible in /dev/block during the early_command stage?
Or, is there any other way to get the correct partitions on the desired arrays?
Background
I have to install many servers. Any manual action in the installation process should be automated. We have different types of servers, from different manufacturers. Some servers have different types of disks in them and we use the RAID controllers to create arrays (Dell, for example, calls them "Virtual Disks", other manufacturers refer to them as "Logical disks", etc. I call them arrays). Our relational database machines are designed to have three arrays:
We use Redfish to create those RAID arrays. I will leave out the details of how we do that, but that works.
The RAID arrays are created in the order as listed above.
The problem
Even though we create the arrays in the order we like them, they're not configured in the desired order. And that order is not predictable. So, if we use /dev/sda, /dev/sdb and /dev/sdc, we may end up with our OS+binlogs volume on the spinning backup disks, while our DB data volume is written on the 900G SSD's and the backups on the SSD's intended for data. (or any variant you can think of). In the images below you'll another example of what we don't want:
![Image]()
![Image]()
What I tried to circumvent this:
We use Dell Poweredge r660xs and Fujitsu Primergy rx2530M7s servers. Both are shipped with Broadcom RAID controllers which work nearly identical.
Details about loading the megaraid_sas driver:
I added this to the early_command script:
[tl;dr] Please advice me on how to automatically figure out which disk (actually RAID array) is which during the early_command stage of an automated install while the RAID controller presents the disks to the kernel in random order and I want to use specific arrays for specific logical volumes.
At this moment, my question is: How can I make the arrays visible in /dev/block during the early_command stage?
Or, is there any other way to get the correct partitions on the desired arrays?
Background
I have to install many servers. Any manual action in the installation process should be automated. We have different types of servers, from different manufacturers. Some servers have different types of disks in them and we use the RAID controllers to create arrays (Dell, for example, calls them "Virtual Disks", other manufacturers refer to them as "Logical disks", etc. I call them arrays). Our relational database machines are designed to have three arrays:
- OS+binlogs (SSD storage ~900G)
- DB data (SSD storage ~3,5T)
- Backups (HDD storage ~4T)
We use Redfish to create those RAID arrays. I will leave out the details of how we do that, but that works.
The RAID arrays are created in the order as listed above.
The problem
Even though we create the arrays in the order we like them, they're not configured in the desired order. And that order is not predictable. So, if we use /dev/sda, /dev/sdb and /dev/sdc, we may end up with our OS+binlogs volume on the spinning backup disks, while our DB data volume is written on the 900G SSD's and the backups on the SSD's intended for data. (or any variant you can think of). In the images below you'll another example of what we don't want:
What I tried to circumvent this:
- The RAID controller assigns ID's to the arrays. For a moment we thought these affected how the controller presented the arrays to the driver. The ID's were assigned in decreasing order, so I created the arrays in opposite order. No luck, obviously.
- Dell seems to support a Firmware Device Order option (which is about the only thing you cannot enable using RedFish!) which promises to address exactly this problem. We tried and proved it's not working.
- I tried to add some code to the early_command script we have. That code could use the `/dev/block/sd?/queue/rotational` and `/dev/block/sd?/size` information to determine which disk is for what. I reasoned: the smallest SSD is always for the OS, the biggest for DB data and the rotating one for backups. Sadly, despite writing a beautiful script (see below), the megaraid_sas driver isn't loaded yet so `/dev/block` is still empty at this stage. I then tried to download a megaraid_sas.ko file to load but that fails as well. (I made sure it's the same kernel as busybox is running on then, but this is were I have not enough knowledge of the process to fix the load issue).
- I asked a similar question to this one in the Dell community forum where I focus on presenting the arrays to the OS in the right order. This question is about my alternative approach: How can I fix this manufacturer-independent in the Debian Installer?
We use Dell Poweredge r660xs and Fujitsu Primergy rx2530M7s servers. Both are shipped with Broadcom RAID controllers which work nearly identical.
Details about loading the megaraid_sas driver:
I added this to the early_command script:
wget -q $recipe_url_base/megaraid_sas.koinsmod megaraid_sas.koGives
insmod: ERROR: could not insert module megaraid_sas.ko: Invalid module format.I checked and busybox runs:
Linux hostname 6.1.0-22-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1and the driver I tried came from an installed Debian server:
/lib/modules/6.1.0-22-amd64/kernel/drivers/scsi/megaraid/megaraid_sas.koI made sure that scsi_mod was loaded already. (megaraid_sas depends on that)
Statistics: Posted by aswen — 2024-07-05 11:12 — Replies 0 — Views 5