5. Configuration

5.1. Virtual Server Configuration file

There’s one central virtual server configuration file which is ~/.infrasim/.node_map/default.yml (source code). All adjustable parameters are defined in this file. This is the only file to modify if you want to customize or make adjustment on the virtual server node. While not all supported options are explicitly listed in this file for purpose of simplicity. However there’s one example configuration file - /etc/infrasim.full.yml.example (source code) - listed all supported parameters and definitions. By referring content in example file, you can update node configuration and then restart infrasim node service and then new properties will take effect.

Here’s steps for this example:

# Operating against node with configuration update
sudo infrasim node destroy <node>

# Edit node configuration
sudo infrasim config edit <node>
# or update a yaml file to the node
sudo infrasim config update <node> <updated_yml>

# Start with new configuration
sudo infrasim node start <node>

Caution

To load the newly updated configuration, you must destroy runtime instance, then start this node again.

If you want to manage your own configuration and start infrasim instance accordingly, refer to this article: Manage Node Config

Here’s full list of the example configuration file; every single key-value pair is supported to be add/modify in your real-in-use infrasim.yml:

# This example virtual server configuration file intends to throughout
# list parameters and properties that infrasim-compute virtual server
# supports to adjust. In most cases it is fine to use default value
# for particuar configuration by skipping putting it into infrasim.yml
# configuration file. For anything item you're interested, it is recommended
# to look up infomation here first. For example, if you'd like to customize
# properties of your drive - either serial number or vender - in below there're
# corresponding item to show how to achieve that.

# Unique identifier
name: node-0

# Node type is mandatory
# Node type of your infrasim compute, this will determine the
# bmc emulation data and bios binary to use.
# Supported compute node names:
#  quanta_d51
#  quanta_t41
#  dell_c6320
#  dell_r630
#  dell_r730
#  dell_r730xd
#  s2600kp - Rinjin KP
#  s2600tp - Rinjin TP
#  s2600wtt - Node of Hydra, Python
type: quanta_d51
namsespace: spans

compute:
    boot:
        # n - Network (PXE);
        # c - hard disk;
        # d - cdrom;
        boot_order: ncd
        menu: on
        splash: <path/to/your splash picture>
        splash-time: 30000
    cdrom:
      file: /home/ubuntu/seed.iso
    kvm_enabled: true
    numa_control: true
    # extra_option can be used to extend qemu command which infrasim did not support yet.
    # an example: -msg timestamp[=on|off]
        # change the format of messages
        # on|off controls leading timestamps (default:on)
    extra_option: -msg timestamp=on
    kernel: /home/infrasim/vmlinuz-3.16.0-25-generic
    initrd: /home/infrasim/initrd.img-3.16.0-25-generic
    cmdline: API_CB=192.168.1.1:8000 BASEFS=base.trusty.3.16.0-25-generic.squashfs.img OVERLAYFS=discovery.overlay.cpio.gz BOOTIF=52-54-BF-11-22-33
    cpu:
        type: host
        features: +vmx
        quantities: 8
    memory:
        size: 4096
    # Currently the PCI bridge is only designed for megasas storage controller
    # When you create multiple megasas controller, the controllers will be assigned
    # a different pci bus number
    pci_bridge_topology:
        -
            device: i82801b11-bridge
            addr: 0x1e.0x0
            multifunction: on
                -
                    device: pci-bridge
                    chassis_nr: 0x1
                    msi: false
                    addr: 0x1
    storage_backend:
        -
            type: ahci
            max_drive_per_controller: 6
            drives:
                -
                    model: SATADOM
                    serial: HUSMM142
                    bootindex: 1
                    # To boot esxi, please set ignore_msrs to Y
                    # sudo -i
                    # echo 1 > /sys/module/kvm/parameters/ignore_msrs
                    # cat /sys/module/kvm/parameters/ignore_msrs
                    file: chassis/node1/esxi6u2-1.qcow2
                -
                    vendor: Hitachi
                    model: HUSMM0SSD
                    serial: 0SV3XMUA
                    # To set rotation to 1 (SSD), need some customization
                    # on qemu
                    # rotation: 1
                    # Use RAM-disk to accelerate IO
                    file: /dev/ram0
                -
                    vendor: Samsung
                    model: SM162521
                    serial: S0351X2B
                    # Create your disk image first
                    # e.g. qemu-img create -f qcow2 sda.img 2G
                    file: chassis/node1/sda.img
                    page_file: chassis/node1/samsung1.bin
                -
                    vendor: Samsung
                    model: SM162521
                    serial: S0351X3B
                    file: chassis/node1/sdb.img
                    page_file: chassis/node1/samsung2.bin
                -
                    vendor: Samsung
                    model: SM162521
                    serial: S0451X2B
                    file: chassis/node1/sdc.img
                    page_file: chassis/node1/samsung3.bin
        -
            type: megasas-gen2
            use_jbod: true
            use_msi: true
            max_cmds: 1024
            max_sge: 128
            max_drive_per_controller: 1
            drives:
                -
                    vendor: Hitachi
                    product: HUSMM168XXXXX
                    serial: SN0500010351XXX
                    rotation: 1
                    slot_number: 0
                    wwn: 0x50000ccaxxxxxxxx
                    file: <path/to/your disk file>
                    page_file: <path/to/your page bin file>

    networks:
        -
            network_mode: bridge
            # Bridge need to be prepared beforehand with brctl
            network_name: br0
            device: vmxnet3
            mac: 00:60:16:9e:a8:e9
        -
            network_mode: nat
            network_name: ens160
            device: e1000
    ipmi:
        interface: bt
        chardev:
            backend: socket
            host: 127.0.0.1
            reconnect: 10
        ioport: 0xca8
        irq: 10
    smbios: chassis/node1/quanta_d51_smbios.bin
    monitor:
        mode: readline
        chardev:
            backend: socket
            server: true
            wait: false
            host: 127.0.0.1
            port: 2345
    # set vnc display <X>
    vnc_display: 1
bmc:
    interface: br0
    username: admin
    password: admin
    address: <ip address>
    channel: 1
    lancontrol: <path/to/lan control script>
    chassiscontrol: <path/to/chassis control script>
    startcmd: <cmd to be excuted>
    startnow: true
    poweroff_wait: 5
    kill_wait: 5
    historyfru: 20
    config_file: <path/to/your config file>
    emu_file: chassis/node1/quanta_d51.emu
    ipmi_over_lan_port: 623

# racadm is a segment of attributes defined only for dell server
racadm:
    # Network to start racadm service
    interface: br0
    port: 10022
    # Credential to access
    username: admin
    password: admin
    # Temporary data provider
    data: /home/infrasim/racadm_data

# SSH to this port to visit ipmi-console
ipmi_console_ssh: 9300

# Renamed from telnet_listen_port to ipmi_console_port, extracted from bmc
# ipmi-console talk with vBMC via this port
ipmi_console_port: 9000

# Used by ipmi_sim and qemu
bmc_connection_port: 9100

# Socket file to bridge socat and qemu
serial_socket: /tmp/serial

Up to infrasim-compute commit a02417c3

  • name

    This attribute defines nodes name, which is a unique identifier for infrasim-compute instances on the same platform. More specifically, it is used as workspace folder name.

    NOT Mandatory

    Default: “node-0”

    Legal Value: String

  • type

    This attribute defines supported nodes type in InfraSIM. With this attribute, infrasim-compute will set BMC emulation data for ipmi_sim and BIOS binary for qemu accordingly, you can get corresponding .emu and .bin in /usr/local/etc/infrasim/ by default.

    Mandatory

    Legal Values:

    • “quanta_d51”
    • “quanta_t41”
    • “dell_c6320”
    • “dell_r630”
    • “dell_r730”
    • “dell_r730xd”
    • “s2600kp”, for Rinjin KP
    • “s2600tp”, for Rinjin TP
    • “s2600wtt”, for Hydra, Python

  • namespace

    This attribute defines the network namespace where the infrasim-compute instance running in, while there are multiple infrasim-compute instances in one test environment. More specifically, you can refer to repo infrasim-network to setup two-layer vswitches and network namespaces.

    NOT Mandatory

    Default: None.

    Legal Value: String

  • compute

    This block defines all attributes used by QEMU. They will finally be translated to one or more qemu command options. The module infrasim.model.CCompute is handling this translation. This is much like a definition for libvert, but we may want it to be lite, and compatible with some customized qemu feature in InfraSIM.

  • compute:boot

    This group of attributes set qemu boot characteristics. See -boot in qemu-doc.

  • compute:boot:boot_order

    This attribute defines boot order for qemu. Will be translated to -boot {boot_order}.

    Not Mandatory

    Default: “ncd”, means in a order of pxe > disk > cdrom.

    Legal Value: See -boot in qemu-doc.

  • compute:boot:menu

    This attribute can enable interactive boot menus/prompts via menu=on as far as firmware/BIOS supports them. If menu=on is set and the firmware/BIOS supports boot menus, the interactive boot menu will be shown when press the shortcuts according to the hint message at boot time. Here is a bios file which supports interactive boot menus.

    Here is a command line to check whether the bios can support menu or not:

    # boot with an interactive boot menu with 20-second splash time and the bios file "bios.bin"
qemu-system-x86_64 -boot menu=on,splash-time=20000 -bios bios.bin

    Perform infrasim init, then this bios file will be downloaded and saved in /usr/local/share/qemu/bios-256k.bin as InfraSIM default bios file.

    Not Mandatory

    Default: None, means non-interactive boot, and there will be no menu=on or menu=off option.

    Legal Value: on or off.

  • compute:boot:splash

    This attribute defines the splash picture path. This picture will be passed to bios, enabling user to show it as logo. This splash file could be a jpeg file or a BMP file in 24 BPP format(true color). The resolution should be supported by the SVGA mode, so the recommended is 320x240, 640x480, 800x640.

    Not Mandatory

    Default: None.

    Legal Value: a valid file path, absolute or relative.

  • compute:boot:splash-time

    This attribute defines the splash time.

    Not Mandatory

    Default: None, means splash time is 0.

    Legal Value: positive integer. 30000 means 30 seconds.

  • compute:kvm_enabled

    This attribute enable kvm when you announce it as True and your system supports kvm. It will be translated to --enable-kvm. You can check if your system supports kvm by check if /dev/kvm exists.

    Not Mandatory

    Default: Depends on if /dev/kvm exists.

    Boolean Table

    kvm_enabled /dev/kvm –enable-kvm
    true yes yes
    true no no
    false yes no
    false no no
    not define yes yes
    not define no no

  • compute:numa_control

    This attribute enable NUMA to improve InfraSIM performance by binding to certain physical cpu. If you have installed numactl and set this attribute to True, you will run qemu in a way like numactl --physcpubind={cpu_list} --localalloc.

    Not Mandatory

    Default: Disabled

  • compute:kernel

    This attribute specifies the binary kernel file path. It will be used by qemu to install.

    Not Mandatory

    Default: None.

  • compute:initrd

    This attribute specifies the initial ram disk path. This INITRD image can be used to provide a place for qemu to install kernel. See -initrd file in qemu-doc.

    Mandatory: depends on if kernel is given.

    Default: None.

  • compute:cmdline

    This attribute will be appended to qemu in string as part of the option --append {cmdline}. See --append in qemu-doc. It will be then used by qemu as kernel parameters. You can view your O/S’s kernel parameters by cat /proc/cmdline.

    Not Mandatory

    Default: None, there will be no --append option.

  • compute:cpu

    This group of attributes set qemu cpu characteristics. The module infrasim.model.CCPU is handling the information.

  • compute:cpu:model

    This attribute sets qemu cpu model.

    Not Mandatory

    Default: “host”

    Legal Values: See -cpu model in qemu-doc.

  • compute:cpu:features

    This attribute adds or removes cpu flags according to your customization. It will be translated to -cpu Haswell,+vmx for example.

    Not Mandatory

    Default: “+vmx”

    Legal Values: See -cpu model in qemu-doc.

  • compute:cpu:quantities

    This attribute sets virtual cpu numbers in all. With default socket 2, CCPU calculates core per socket. Default set to 1 thread per cores. It will be translated to -smp {cpus},sockets={sockets},cores={cores},threads=1 for example.

    Not Mandatory

    Default: 2

    Legal Values: See -smp in qemu-doc.

  • compute:memory

    This attribute refers to RAM, which the virtual computer devices use to store information for immediate use. The module infrasim.model.CMemory is handling the information.

  • compute:memory:size

    This attribute sets the startup RAM size. The default is 1024MB.

    Default: 1024

    Legal Values: See -m in qemu-doc.

  • compute:storage_backend

    This block defines backend storage details. It maintains a list of controller structures, and each controller maintains a list of drive structures.

  • compute:storage_backend:-

    Each element of this list defines a storage controller, they have some common attributes. The module infrasim.model.CBaseStorageController is handling the information. Developer may inherits this class to define other type of controller and specific controller attributes.

    Common attributes:

    • type
    • max_drive_per_controller

    Specific controllers defined:

    Controller Type Module Attributes
    megasas.* infrasim.model.MegaSASController use_jbod sas_address use_msi max_cmds max_sge
    lsi.* infrasim.model.LSISASController  
    .*ahci.* infrasim.model.AHCIController  

  • compute:storage_backend:-:type

    Define types of a controller, this makes infrasim-compute model handle other attributes accordingly.

  • compute:storage_backend:-:max_drive_per_controller

    This is a protection mechanism that you write too much in drives list. If the actual count of drives exceeds this limitation, infrasim-compute now make more controller, in the same attribute but different PCI bus number, to mount all drives. The module infrasim.model.CPCITopologyManager defines this logic.

  • compute:storage_backend:-:controller:drives

    This attribute defines a list of drives mounted on the controller. Common attributes are managed by infrasim.model.CBaseDrive. Developer may inherits this class to define other type of drive and specific attributes.

    Common attributes - device personality options:

    • bootindex
    • serial
    • wwn
    • version

    Common attributes - simulation options:

    • format
    • cache
    • aio
    • size
    • file
    • page-file

    Drive type currently depends on the controller it is mounted on:

    Controller Type Mounted Drive Type Attributes
    LSISASController MegaSASController infrasim.model.SCSIDrive port_index port_wwn channel scsi-id lun slot_number product vendor rotation
    AHCIController infrasim.model.IDEDrive model

  • compute:storage_backend:-:controller:drives:-:bootindex

    Cite from qemu’s bootindex documentation.

    Block and net devices have bootindex property. This property is used to determine the order in which firmware will consider devices for booting the guest OS. If the bootindex property is not set for a device, it gets lowest boot priority. There is no particular order in which devices with unset bootindex property will be considered for booting, but they will still be bootable.

    NOT Mandatory

    Legal Value: integer

    Example: Let’s assume we have a QEMU machine with two NICs (virtio, e1000) and two disks (IDE, virtio):

    qemu -drive file=disk1.img,if=none,id=disk1

    -device ide-drive,drive=disk1,bootindex=4 -drive file=disk2.img,if=none,id=disk2 -device virtio-blk-pci,drive=disk2,bootindex=3 -netdev type=user,id=net0 -device virtio-net-pci,netdev=net0,bootindex=2 -netdev type=user,id=net1 -device e1000,netdev=net1,bootindex=1

    Given the command above, firmware should try to boot from the e1000 NIC first. If this fails, it should try the virtio NIC next; if this fails too, it should try the virtio disk, and then the IDE disk.

  • compute:storage_backend:-:controller:drives:-:serial

    Drive’s serial number.

    NOT Mandatory

  • compute:storage_backend:-:controller:drives:-:wwn

    Refer to WWN (wikipedia).

    NOT Mandatory

  • compute:storage_backend:-:controller:drives:-:version

  • compute:storage_backend:-:controller:drives:-:format

    Cite from QEMU:

    Specify which disk format will be used rather than detecting the format. Can be used to specifiy format=raw to avoid interpreting an untrusted format header.

    This attribute will be translated to -drive format={format}.

  • compute:storage_backend:-:controller:drives:-:cache

    Cite from QEMU:

    cache is “none”, “writeback”, “unsafe”, “directsync” or “writethrough” and controls how the host cache is used to access block data.

    This attribute will be translated to -drive cache={cache}.

  • compute:storage_backend:-:controller:drives:-:aio

    Cite from QEMU:

    aio is “threads”, or “native” and selects between pthread based disk I/O and native Linux AIO.

    This attribute will be translated to -drive aio={aio}.

  • compute:storage_backend:-:controller:drives:-:file

    Cite from QEMU:

    This option defines which disk image to use with this drive.

    This attribute will be translated to -drive file={file}.

  • compute:storage_backend:-:controller:drives:-:page-file

    This option allows user to specify drive page data, which can provide addtional information for client OS, including mode sense pages and inquiry data pages. The page file is generated by a tool which can fetch data from HW drive or user defined json file.

    Command, e.g. sudo python gen_page_utility.py -d /dev/sdb -o drive_name.bin, will create a drive page bin file.

    For more details, please refer to how-to-generate-drive-page-files.

    This attribute will be translated to -device page_file={file}.

  • compute:storage_backend:-:controller:drives:-:size

    If infrasim-compute application can’t detect existing drive file, it will help user create a drive image file. A command, e.g. qemu-img create -f qcow2 sda.img 10G, will be called to create such a drive file in node workspace. This is where size take effects.

    Not Mandatory

    Default: 8

    Legal Values: integer, in unit of GB

  • compute:networks
  • compute:networks:-:network_mode
  • compute:networks:-:network_name
  • compute:networks:-:device
  • compute:networks:-:mac
  • compute:ipmi
  • compute:ipmi:interface
  • compute:ipmi:chardev
  • compute:ipmi:chardev:backend
  • compute:ipmi:chardev:host
  • compute:ipmi:chardev:reconnect
  • compute:ipmi:ioport
  • compute:ipmi:Irq
  • compute:smbios
  • compute:monitor
  • compute:monitor:mode
  • compute:monitor:chardev
  • compute:monitor:chardev:backend
  • compute:monitor:chardev:server
  • compute:monitor:chardev:wait
  • compute:monitor:chardev:path
  • compute:vnc_display

  • compute:cdrom

    This attribute specify a media when qemu boot from cdrom. You can promote cdrom boot order by specify d first in compute:boot:boot_order.

    Not Mandatory

    Legal Values: path to a image file, or directly use cdrom device, e.g. /dev/sr0

  • bmc

    This block defines attributes used by OpenIPMI. They will finally be translated to one or more ipmi_sim command options, or be defined in the configuration file for it. The module infrasim.model.CBMC is handling this translation.

  • bmc:interface

    This attributes defines both:

    • from which network ipmi_sim will listen IPMI request
    • BMC’s network properties printed by ipmitool lan print

    The module infrasim.model.CBMC takes this attribute and comes out with two variable defined in ipmi_sim configuration template.

    • {{lan_interface}}, network name for ipmitool lan print to print, e.g. “eth0”, “ens190”.
    • {{ipmi_listen_range}}, IP address that ipmi_sim shall listen to and response IPMI command. If you set a valid interface here, an IP address in string will be assigned to this variable, e.g. “192.168.1.1”.

    Not Mandatory

    Default

    • {{lan_interface}}: first network device except lo.
    • {{ipmi_listen_range}}: ”::”, so that you shall see addr :: 623 in vbmc.conf, it means ipmi_sim listen to IPMI request on all network on port 623

    Valid Interface: Use network devices from ifconfig.

    • {{lan_interface}}: the specified network interface.
    • {{ipmi_listen_range}}: IP address of lan_interface(“0.0.0.0” if interface has no IP).

    Invalid Interface: Network devices that don’t exist.

    • {{lan_interface}}: no binding device
    • {{ipmi_listen_range}}: no range setting, which means user could only access ipmi_sim through kcs channel inside qemu OS.
  • bmc:username
  • bmc:password
  • bmc:address
  • bmc:channel
  • bmc:lancontrol
  • bmc:chassiscontrol
  • bmc:startcmd
  • bmc:startnow
  • bmc:poweroff_wait
  • bmc:historyfru
  • bmc:config_file
  • bmc:emu_file
  • bmc:ipmi_over_lan_port

  • racadm

    This block defines RACADM (Remote Access Controller ADMin) simulation behavior.

  • racadm:interface

    This attribute defines on which interface RACADM shall listen to. It will then start as a service, listening on the certain IP.

    Not Mandatory

    Default: if you don’t set this attribute, RACADM will start listening on 0.0.0.0

    Legal Values: a valid interface with IP address

  • racadm:port

    This attribute defines on which port RACADM shall listen to. It works with the :racadm:interface:yamlRacadmInterface.

    Not Mandatory

    Default: 10022

    Legal Values: a valid port that is not being used

  • racadm:username

    SSH username on RACADM simulation.

    Default: admin

  • racadm:password

    SSH password on RACADM simulation.

    Default: admin

  • racadm:data

    You need to specify a folder name for this attribute, e.g. /home/infrasim/data. In this folder, you need to provide several pure text files. Each file maintains response for a certain RACADM command.

    RACADM simulation now is not getting runtime data from BIOS binary or IPMI emulation data, but using this temporary implementation to inject data for RACADM simulation.

    Here is a list of supporting data and required text file name (without extension .txt).

    RACADM Command Response File Name
    getled getled
    getsysinfo getsysinfo
    storage get pdisks –o storage_get_pdisks_o
    get BIOS get_bios
    get BIOS.MemSettings get_bios_mem_setting
    hwinventory hwinventory
    hwinventory nic.Integrated.1-1-1 hwinventory_nic_integrated_1-1-1
    hwinventory nic.Integrated.1-2-1 hwinventory_nic_integrated_1-2-1
    hwinventory nic.Integrated.1-3-1 hwinventory_nic_integrated_1-3-1
    hwinventory nic.Integrated.1-4-1 hwinventory_nic_integrated_1-4-1
    get IDRAC get_idrac
    setled -l 0 setled_l_0
    get LifeCycleController get_life_cycle_controller
    get LifeCycleController.LCAttributes get_life_cycle_controller_lc_attributes
  • ipmi_console_ssh
  • ipmi_console_port
  • bmc_connection_port

  • serial_socket

    This attribute defines a unix socket file to forward data. More specifically, it bridges socat and qemu for InfraSIM to forward input and output stream as a serial port. With this attribute designed, you will see socat starts with option unix-listen:<file>, while qemu starts with a socket chardev -chardev socket,path=<file>,id=...

    Not Mandatory

    Default: a file named .socket in node workspace

    Legal Values: a valid file path, absolute or relative, to create such node

5.2. Networking

  1. Virtual server NAT or host-only mode, this is default mode implemented in infrasim-compute
    • vCompute is accessible ONLY inside Ubuntu host
    • Software running in vCompute can access outside network if connecting Ubuntu host NIC with virtual bridge
    • Configuration YAML file can specify which NIC IPMI over LAN traffic flows through
    _images/networking_nat.PNG
  2. Bridge mode - single
    • Work as virtual switch
    • Connect BMC NIC and NICs in virtual compute together
    • Configuration YAML file controls how many NICs that virtual compute has and specify bridge they connect to
    _images/networking_bridge_single.PNG

    Note

    It requires setting up bridge and connect to NIC of underlying host in advance.

    Here’s steps for this example:

    # brctl addr br0
# brctl addif br0 eth1
# brctl setfd br0 0
# brctl sethello < bridge name > 1
# brctl stp br0 no
# ifup br0

  3. Bridge mode - multiple

    _images/networking_bridge_multiple.PNG