Andrew Latham

Note to self, a full post on IPMITool would be good.

When on-boarding new hardware always configure the IPMI devices properly. You can use tools to interface with the IPMI devices and configure defaults that will add local administrator accounts. Tools like IPMITools have options to download existing settings or upload/set new settings. Tools like OpenStack [2] Ironic and various other stacks are enabling this in fantastic ways. Don't limit access to systems management devices with fear, enable access so that team members can get useful information quickly. An example of why you want to share access would be:

ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 sensor
CPU Core1 Temp   | 76.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 98.000    | 100.000                                                                                                              
CPU Core2 Temp   | 76.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 98.000    | 100.000                                                                                                              
CPU SoC Temp     | 75.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 98.000    | 100.000                                                                                                              
System Temp      | 23.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000                                                                                                               
Peripheral Temp  | 24.000     | degrees C  | ok    | -9.000    | -7.000    | -5.000    | 80.000    | 85.000    | 90.000                                                                                                               
FAN 1            | 1600.000   | RPM        | ok    | 400.000   | 576.000   | 784.000   | 33856.000 | 34225.000 | 34596.000                                                                                                            
FAN 2            | na         |            | na    | na        | na        | na        | na        | na        | na                                                                                                                   
FAN 3            | na         |            | na    | na        | na        | na        | na        | na        | na                                                                                                                   
Vcore            | 0.992      | Volts      | ok    | 0.776     | 0.800     | 0.824     | 1.352     | 1.376     | 1.400                                                                                                                
VDIMM            | 1.568      | Volts      | ok    | 1.288     | 1.312     | 1.336     | 1.656     | 1.680     | 1.704                                                                                                                
+5 V             | 5.024      | Volts      | ok    | 4.416     | 4.448     | 4.480     | 5.536     | 5.568     | 5.600                                                                                                                
+5VSB            | 4.992      | Volts      | ok    | 4.416     | 4.448     | 4.480     | 5.536     | 5.568     | 5.600     
+12 V            | 12.296     | Volts      | ok    | 10.600    | 10.653    | 10.706    | 13.250    | 13.303    | 13.356    
+3.3 V           | 3.288      | Volts      | ok    | 2.880     | 2.904     | 2.928     | 3.648     | 3.672     | 3.696     
+3.3VSB          | 3.264      | Volts      | ok    | 2.880     | 2.904     | 2.928     | 3.648     | 3.672     | 3.696     
VBAT             | 0.624      | Volts      | nr    | 2.880     | 2.904     | 2.928     | 3.648     | 3.672     | 3.696     
+1.05 V          | 1.072      | Volts      | ok    | 0.808     | 0.816     | 0.824     | 1.264     | 1.288     | 1.312     
Chassis Intru    | 0x0        | discrete   | 0x0000| na        | na        | na        | na        | na        | na        
PS Status        | 0x1        | discrete   | 0x0100| na        | na        | na        | na        | na        | na   

or

# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 sdr
CPU Core1 Temp   | 76 degrees C      | ok
CPU Core2 Temp   | 76 degrees C      | ok
CPU SoC Temp     | 74 degrees C      | ok
System Temp      | 22 degrees C      | ok
Peripheral Temp  | 24 degrees C      | ok
FAN 1            | 1600 RPM          | ok
FAN 2            | no reading        | ns
FAN 3            | no reading        | ns
Vcore            | 0.99 Volts        | ok
VDIMM            | 1.57 Volts        | ok
+5 V             | 5.02 Volts        | ok
+5VSB            | 4.99 Volts        | ok
+12 V            | 12.30 Volts       | ok
+3.3 V           | 3.29 Volts        | ok
+3.3VSB          | 3.26 Volts        | ok
VBAT             | 0.62 Volts        | nr
+1.05 V          | 1.07 Volts        | ok
Chassis Intru    | 0x00              | ok
PS Status        | 0x01              | ok

In this case I have a Supermicro system where I have an account configured for my normal username and I have rights to administer the device. After the users are added then the default username can have the password changed to a more secure default or removed. On-boarding is an ordered process and software is here to help us do these ordered processes over and over again.

If you are interested in the details you can read the specs on new IPMI devices at Intel [3] for example. Section 22.30 will show you how the system deals with passwords for example.

ipmitool -I lan -U ADMIN -H host-ipmi.domain.net user set name 3 operations
ipmitool -I lan -U ADMIN -H host-ipmi.domain.net channel setaccess 1 3 link=on ipmi=on privilege=4

Older IPMI only handle 16 char passwords

ipmitool -I lan -U ADMIN -H host-ipmi.domain.net user set password 3 16 abcdefghijklmnop

New 2.0 IPMI handles 20 char passwords

ipmitool -I lan -U ADMIN -H host-ipmi.domain.net user set password 3 20 abcdefghijklmnopqrst

Lastly Enable

ipmitool -I lan -U ADMIN -H host-ipmi.domain.net user enable 3

Complete example with output

# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 user set name 5 operations
# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 user set password 5 HardPassword
Set User Password command successful (user 5)
# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 channel setaccess 1 5 privilege=4 link=on ipmi=on
Set User Access (channel 1 id 5) successful.
# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 user test 5 16 HardPassword
Success
# ipmitool -I lan -U lathama -f ipmifile -H 192.168.15.206 user enable 5
#

Keep in mind there is a huge amount of options an all are very important. It is worth while to review the information in detail.

  1. https://sourceforge.net/projects/ipmitool/
  2. https://wiki.openstack.org/wiki/Ironic
  3. https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/ipmi-intelligent-platform-mgt-interface-spec-2nd-gen-v2-0-spec-update.pdf
Andrew Latham

On systems like Debian Stretch with systemd the time sync is baked in but will not run if the legacy NTP package is installed at all so here is a howto/demo of what to do.

# apt-get purge ntp
# systemctl restart systemd-timesyncd.service
# systemctl status systemd-timesyncd.service 
● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
           └─disable-with-time-daemon.conf
   Active: active (running) since Fri 2017-08-11 10:09:11 CDT; 3s ago
     Docs: man:systemd-timesyncd.service(8)
 Main PID: 31413 (systemd-timesyn)
   Status: "Synchronized to time server 92.243.6.5:123 (0.debian.pool.ntp.org)."
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/systemd-timesyncd.service
           └─31413 /lib/systemd/systemd-timesyncd

Aug 11 10:09:11 nodeone systemd[1]: Starting Network Time Synchronization...
Aug 11 10:09:11 nodeone systemd[1]: Started Network Time Synchronization.
Aug 11 10:09:11 nodeone systemd-timesyncd[31413]: Synchronized to time server 92.243.6.5:123 (0.debian.pool.ntp.org).
#
Andrew Latham

Clean installs directly from the repos...

Using libvirt (virsh/virt-install/virt-manager) you can install from the HTTP repo of a Linux distribution to be super lazy. During installation you simply provide the URI in the location field or tools like virt-manager will have an option in the wizard.

For Debian you would use:

http://deb.debian.org/debian/dists/stable/main/installer-amd64/

For CentOS which lacks a CDN mirror or I could not find it quickly, you can try:

http://mirrors.kernel.org/centos/7/os/x86_64/
Andrew Latham

Devops without the extra parts

You want to deploy code onto a server into a certain directory. You have SSH to the server and it has Git installed. Your desired destination is /home/user/public_html/production/

Andrew Latham

Easy start with Ansible

Example from a local ansible source tree without using any install to run adhoc commands

Setup

Download or checkout the software, unpack and change into the base directory. We will then run a script to setup the environment. This can and should be done as a user and not root. Note you may need to install packages like python3-paramiko, python-paramiko, python3-jinja2, python-jinja2, python3-yaml, python-yaml and others to use Ansible.

cat hacking/README.md
source hacking/env-setup

Assume key works

./bin/ansible all -i 192.168.15.11, -a "uname -a"
192.168.15.11 | SUCCESS | rc=0 >>
Linux nodeone 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 GNU/Linux

Set key

 ./bin/ansible all -i 192.168.15.11, -a "uname -a" --private-key=~/.ssh/id_rsa
192.168.15.11 | SUCCESS | rc=0 >>
Linux nodeone 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 GNU/Linux

whoami

./bin/ansible all -i 192.168.15.11, -a "whoami" --private-key=~/.ssh/id_rsa
192.168.15.11 | SUCCESS | rc=0 >>
lathama

become root via su

./bin/ansible all -i 192.168.15.11, --private-key=~/.ssh/id_rsa -b --become-method=su -K -a "whoami"
SU password: 
192.168.15.11 | SUCCESS | rc=0 >>
root
Andrew Latham

Using tools like libvirt, virt-manager to network boot (PXE) systems.

TL;DR;

mkdir -p /srv/tftp
cp -r your_pxelinux_stuffs /srv/tftp/
virsh net-edit default

replace

<dhcp>

with

<tftp root='/srv/tftp'/>
<dhcp>
  <bootp file='pxelinux.0'/>

then

virsh net-destroy default && virsh net-start default

Profit