| Service | Code name | Description | Core Tech | Port |
|---|---|---|---|---|
| Identity Service | Keystone | User Management | authentication and authorization | 35357(adminurl) & 5000(publicurl) |
| Compute Service | Nova | Virtual Machine Management | KVM, Qemu,Libvirtd | |
| Image Service | Glance | Manages kernel image or disk image | 9191 & 9292 | |
| Dashboard | Horizon | UI Dashboard for OpenStack Management | ||
| Object Storage | Swift | Provides Cloud Object Storage(s3) | 6000,6001,6002 | |
| Block Storage | Cinder | Block Storage | LVM,CEPH | |
| Network Service | Neutron | Networking service | OpenVswitch,Linux Bridge | 9696 |
| Load Balancing | Octavia | LB service | HAPROXY,keepalived,VRRP | |
| Key Manager Service | Barbican | Key service |
| Service | Code name |
|---|---|
| TAP device | A tap device is a software-only interface, that a userspace program can attach to itself and send/receive packets to it. TAP devices are they way that KVM/QEMU implement virtual NIC (vNIC) attached to the VMs , A TUN/TAP device is a virtual device where one end is a kernel network interface, and the other end is a user space file descriptor. |
| veth pair | veth pair is a pair of virtual NIC cards connected via a virtual cable. If a packet is sent on one of them, it will come out of the other one and vise versa. veth are usually used to connect two entities. |
| Linux bridge | Linux bridge is a virtual switch implemented in Linux |
| Openvswitch | openvswitch is a more complicated vritual switch implemented in Linux. It allows openflow rules to be applied to traffic at layer 2 such that decisions are made on MAC addresses, VLAN ID of the traffic flow. Openvswitch provides native support for VXLAN tunnels |
| Network namespaces | An isolated network stack in Linux, where you can have isolated interfaces, routing tables and iptables rules. Network name spaces do not “see” each other’s traffic. This is vital for openstack since you let your users create their own VM networks and you need this level of isolation at layer |
| Patch interfaces in openvswitch | a special kind of interface that is used to connect two openvswitch switches e.g patch-tun –> patch-int |
| neutron-where-did-my-packets-go |
| Name | Command |
|---|---|
| Official Cheatsheet | OpenStack cli-cheat-sheet |
| OpenStack Glossary | OpenStack Glossary |
| Issue keystone Token | openstack token issue -f value -c id |
| identity provider list. | openstack identity provider list |
| Cinder Service list | cinder service-list |
| Compute service list | openstack compute service list |
| Neutron agent list | openstack network agent list |
| Hypervisor list | openstack hypervisor list |
| Aggregate list | openstack aggregate list |
| Availability zone List | openstack availability zone list |
| Project list | openstack project list |
| Project Quota | openstack quota show $TENANT_NAME |
| Enable Compute service on specific Hypervisor | openstack compute service set --enable $COMPUTE_NAME nova-compute |
| Show n/w details | openstack network show $NETWORK_NAME |
| Show subnet details | openstack subnet show $SUBNET_NAME |
| Set quota on router | openstack quota set --routers 3 $ROUTER_ID |
| Set quota on subnet | openstack quota set --subnets 25 $SUBNET_ID |
| rbac list | neutron rbac-list |
| list openstack services | openstack service list |
| cinder list | cinder list --all|grep kube|grep avail|awk '{print $2}'|xargs -L1 cinder delete |
| Show LB quota | openstack loadbalancer quota show id |
| List user | openstack user list |
| List ERROR instances | openstack server list --all-projects --status ERROR -c ID |
| list all tenant volumes | cinder list --all-tenants |
| list all volumes of specific tenant | cinder --os-tenant-name $TENANT_NAME list |
| Increase Instance Quota for a project | openstack quota set --instances 50 $TENANT_NAME |
| Check all Instance on a specific Hypervisor | openstack server list --all-projects --host $COMPUTE_NAME |
| List IP availability for project | openstack ip availability list --project $TENANT_NAME |
| Launch a Instance on specific Hypervisor | nova boot --flavor 10 --image $IMAGE_NAME --nic net-id=$NETWORK_NAME --availability-zone nova:COMPUTE_NAME $INSTANCE_NAME |
| List Instance by Project | openstack server list --all-projects --project $TENANT_NAME |
| Find Instance Project | openstack server list --all-projects | grep -i $INSTANCE_NAME |
| Console Logs | openstack console log show $INSTANCE_ID |
| Create Floating IP | openstack floating ip create --project $TENANT_NAME --subnet $SUBNET_NAME $NETWORK_NAME |
| Show network IP availability details | openstack ip availability show $NETWORK_NAME |
| List IP availability for networks | openstack ip availability list |
| List L3 agent | openstack network agent list --agent-type l3 |
| List openVswitch Agent | openstack network agent list --agent-type open-vswitch |
| Resource check | openstack resource provider list |
| List OpenStack server by IP | openstack server list --ip 192.168.0.10 OR openstack server list --all-projects --ip 192.168.0.10 |
| Resource detaild view | openstack resource provider show --allocations $COMPUTE_ID -f json |
| Detach a volume from an instance | openstack server remove volume $INSTANCE_ID $VOLUME_ID |
| Check role assignment | openstack role assignment list --name --project $PROJECT_ID |
| Nagios directory | /usr/share/check-mk-agent |
| Name | Command |
|---|---|
| HTTPD | systemctl status httpd |
| MEMCACHED | systemctl status memcached |
| HAPROXY | systemctl status haproxy |
| KEEPALIVED | systemctl status keepalived |
| MARIADB | systemctl status mariadb |
| RABBITMQ | systemctl status rabbitmq-server |
| GLANCE | systemctl status openstack-glance-api openstack-glance-registry |
| NOVA | systemctl status openstack-nova-api openstack-nova-consoleauth openstack-nova-conductor openstack-nova-scheduler openstack-nova-novncproxy |
| NEUTRON | systemctl status neutron-server |
| Name | Command |
|---|---|
| NEUTRON | systemctl status neutron-dhcp-agent neutron-l3-agent neutron-metadata-agent neutron-openvswitch-agent |
| Name | Command |
|---|---|
| NOVA | systemctl status openstack-nova-compute openvswitch neutron-openvswitch-agent |
| Name | Command |
|---|---|
| OCTAVIA | systemctl status octavia-api octavia-worker octavia-health-manager octavia-housekeeping |
| Name | Command |
|---|---|
rabbitmq-diagnostics status | |
rabbitmqctl cluster_status | |
rabbitmqctl list_queues -p openstack | sort -k 2 -g | |
rabbitmqctl list_queues -p openstack name consumers messages | |
| More Info | Clustering Guide |
| Name | Command |
|---|---|
| Check Version | ceph tell mon.* version |
| Check Health Status | ceph health detail |
ceph osd df | |
ceph osd pool ls detail | |
ceph osd pool get rbd all |
| Name | Comment |
|---|---|
| placement | 8778 |
| neutron | 9696 |
| nova | 8774 |
| glance | 9292 |
| keystone | 5000 |
| nova-api port to retrive metadata | 8775 |
| Name | Query | |
|---|---|---|
| retrive metadata | curl http://169.254.169.254/openstack/latest/meta_data.json | |
| retirve userdata | curl 169.254.169.254/1.0/user-data | |
| How OpenStack handels metadata | More on OpenStack Metadata |
| Name | Comment |
|---|---|
/ | |
| == | |
| == | |
| Name | Command |
|---|---|
| List all VM. | openstack server list |
| List all VM for all projects | openstack server list --all-projects |
| Show VM details | openstack server show $INSTANCE_ID |
| Create VM from CLI | openstack server create --flavor 10 --image $IMAGE_ID --key-name $INSTANCE_ID --security-group $SECGRP_ID --nic net-id=$NETWORK_ID $INSTANCE_ID
|
| Get Instance info | openstack server show $INSTANCE_ID |
| Migrate Instance Live on Specific hypervisor | openstack server migrate $UUID --live Hypervisor-moon |
| Migrate Instance Live | openstack server migrate $UUID --live-migration |
| Nova Migration article | VM Migration some-useful-commands |
| Name | Command |
|---|---|
| Octavia glossary | OpenStack official glossary |
| List all LB. | openstack loadbalancer list |
| View LB details | openstack loadbalancer show lb_uuid |
| Create LB | openstack loadbalancer create --name $LB_NAME --vip-subnet-id $SUBNET_ID |
| Create lb listener | openstack loadbalancer listener create --name $LISTENER_NAME --protocol TCP --protocol-port 80 $LB_NAME |
| Show lb listeners details | openstack loadbalancer listener show $LB_ID |
| update lb parameters | openstack loadbalancer listener set --timeout-client-data 60000 --timeout-member-data 60000 $LISTENER_ID |
| create lb pool | openstack loadbalancer pool create --name $POOL_ID --lb-algorithm ROUND_ROBIN --listener $LISTENER_NAME --protocol TCP |
| Add member to lb | openstack loadbalancer member create --subnet-id $SUBNET_ID --address 192.168.0.151 --protocol-port 80 $LB_ID |
| List Amphora node of LoadBalancer | openstack loadbalancer amphora list --loadbalancer $LB_ID |
| List Octavia provider | openstack loadbalancer provider list |
| Find Octavia VM,(only in Service cloud) | openstack server list --name .*octavia.* |
| Find out Error’d Amphora nodes | openstack loadbalancer amphora list | grep -v ALLOCATE |
| List LB pool | openstack loadbalancer pool list |
| List members in pool | openstack loadbalancer member list POOL_ID |
| LB failover | openstack loadbalancer failover $LB_ID |
| LB Delete | openstack loadbalancer delete --cascade $LB_ID |
| Get member Details. | openstack loadbalancer member show $POOL_ID $MEMBER_ID |
| Check all Amphorae on services project | openstack server list --all-projects --project services -c "Availability Zone" -c ID -c Name -c Host -c "Power State" --long |
| Test port Connection from Amphora namespace | ip netns exec amphora-haproxy curl -v telnet://IP:443 |
| Octavia Cookbook | OpenStack official LB Cookbook |
| How to fetch the client IP address at the backend | Fetch the client IP address at the backend |
| insert_headers | openstack loadbalancer listener set --insert-headers X-Forwarded-For true |
| LB statistics | openstack loadbalancer stats show $LB_ID |
| LB Listener statistics | openstack loadbalancer listener stats show $Listener_ID |
| List LB flavors | openstack loadbalancer flavor list |
| Create LB with specific flavor | openstack loadbalancer create --name lb-flavor-test-1 --flavor Octavia_4vCPUx4GB --vip-network-id e33324cc-69d1-4b61-b4b5-9264f7ba0d92 |
| Octavia Features | Octavia Load Balancer Features |
| Name | Command |
|---|---|
| List all Security Group | openstack security group list |
| Create security group e.g ssh | openstack security group create ssh |
| Add a rule to allow ssh and icmp in the ssh security group | openstack security group rule create --ingress --protocol tcp --dst-port 22 ssh |
| Add a rule to allow ssh and icmp in the ssh security group | openstack security group rule create --ingress --protocol tcp --protocol icmp ssh
|
| Name | Command |
|---|---|
| Run Lighweight webserver | while true; do (echo -e 'HTTP/1.1 200 OK\r\n'; echo -e "\n\tMy website has date function" ; echo -e "\t$(date)\n") | nc -lp 8080; done |
| Name | Command |
|---|---|
| Create a glance image webimage using a file centos.qcow2 available inside /tmp | openstack image create --disk-format qcow2 --file /tmp/cinder.qcow2 webimage |
| Download Image from Glance | glance image-download --file CENTOS-8 --progress $IMAGE_ID |
| Show Image details | openstack image show $IMAGE_ID |
| Import/Export of VM Snapshots | OpenStack official import/export Doc
|
| Name | Command |
|---|---|
| Check if IP belong to OpenStack DC | openstack port list | grep $IP_ADDRESS |
| Create VIP Port | openstack port create --project $PROJECT_ID --network $NW_ID --security-group $SEC_ID $VIP_NAME |
| Port set | openstack port set $VM_PORT_ID allowed-address mac-address=$VIP_PORT_ID,ip-address=$VIP_IP |
| Check floating IP | openstack floatingip list |
| disable port_security_enabled flag | openstack port set --disable-port-security $PORT_ID |
| check port in specific project | openstack port list --project $PROJECT_ID |
| List all Security Group | openstack security group list |
| Create security group e.g ssh | openstack security group create ssh |
| Add a rule to allow ssh and icmp in the ssh security group | openstack security group rule create --ingress --protocol tcp --dst-port 22 ssh |
| Add a rule to allow ssh and icmp in the ssh security group | openstack security group rule create --ingress --protocol tcp --protocol icmp ssh
|
| Name | Command |
|---|---|
| Check system metrics of Instance | nova diagnostics $INSTANCE_ID |
| Name | Command |
|---|---|
| List all volumes | openstack volume list --all-project |
| List all snapshot | openstack volume snapshot list --all-project |
| Show snapshot details | openstack volume snapshot show $VOLUME_ID |
| reset volume state | cinder snapshot-reset-state $VOLUME_ID |
| delete volume snapshot | openstack volume snapshot delete $VOLUME_ID |
| Find out snapshot of a volume | openstack volume snapshot list --volume $VOLUME_ID --all-project |
| Force delete cinder volume | cinder force-delete $VOLUME_ID |
| Remove volume from Instance | openstack server remove volume $INSTANCE_ID $VOLUME_ID |
| Add volume to Instance | openstack server add volume $INSTANCE_ID $VOLUME_ID |
| Troubleshooting cinder | Troubleshooting Doc |
| Name | Command |
|---|---|
| Listing Running Services Under SystemD | systemctl --type=service |
| List all active Services | systemctl --type=service --state=active |
| Displaying a Unit File | systemctl cat nginx OR systemctl edit --full nginx.service |
| Check unit files | systemctl list-unit-files |
| Editing Unit Files | systemctl edit nginx.service |
| systemd file location | /etc/systemd/system/ |
| Reload systemd process | systemctl daemon-reload |
| Shows services that failed to run | systemctl --failed |
| see loaded but inactive units | systemctl --failed --all |
| Systemd cheatsheet | Official CheatSheet |
| Name | Command |
|---|---|
| Show all journal entries: | journalctl |
| See boot-related messages from the current boot: | journalctl -b |
| To see the boots that systemd journal knows about | journalctl --list-boots |
| Display all messages generated by process ID | journalctl _PID=1115 |
| Display all messages generated by current user ID (id -u): | journalctl _UID=$(id -u) |
| Display all of the entries that involve the executable in question: | journalctl /bin/bash |
| Displays all messages between two dates: | journalctl --since "2020-07-05 21:30:01" --until "2021-07-05 21:30:02" |
| Printing the last 50 messages logged within the last hour: | journalctl -n50 --since "1 hour ago" |
| Printing the last 50 messages logged within the last minutes: | journalctl -n50 --since "5 minutes ago" |
| json-pretty output | journalctl -u telegraf -f -o json-pretty |
| parse json output | journalctl -u telegraf -f -o json-pretty | jq .MESSAGE |
| Filter logs based on priority | journalctl -p 2 |
| Filter kernel messages | journalctl -k |
| View logs using journalctl in verbose mode | journalctl -o verbose |
| journalctl cheatsheet | Official CheatSheet |
| Name | Command |
|---|---|
| Capture All incoming HTTP POST requests | tcpdump -i eth0 -s 0 -A ‘tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504F5354’ |
| Capture All incoming HTTP GET requests | tcpdump -i eth0 -s 0 -A ‘tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420’ |
| Capturing packets with human readable timestamp | tcpdump -c 8 -tttt -i enp0s3 |
| Capturing and saving packets to a file | tcpdump -w enp0s3-26082018.pcap -i enp0s3 |
| Capturing and Saving the packets whose size greater than N bytes. | tcpdump -w enp0s3-26082018-2.pcap greater 1024 |
| Capturing and Saving the packets whose size less than N bytes | tcpdump -w enp0s3-26082018-3.pcap less 1024 |
| Reading packets from the saved file | tcpdump -r enp0s3-26082018.pcap |
| Find the leaf switch | tcpdump -c 2 -i bond0 -nn "ether[20:2]==0x2000" |
| Find the leaf switch for eth0 | tcpdump -c 2 -i eth0 -nn "ether[20:2]==0x2000" |
| Capture outgoing port traffic | tcpdump -enni tap-interface-id dst port 443 |
| Capture outgoing src and port traffic | tcpdump -enni tap-interface-id src 192.168.0.1 and dstport 443 |
| DNS trace | dig +trace google.com |
| DNS tcpdump | tcpdump -enni eth0 dst port 53 |
| Name | Command |
|---|---|
| List namespaces | ip netns |
| GET namespace PID | ip netns pids |
| Name | Command |
|---|---|
| List active guests | virsh list |
| List all Guests. | virsh list --all |
| Edit Guest | virsh edit GUEST_ID |
| check GUEST xml data | virsh dumpxml GUEST_ID |
| Find the tap interface of VM | virsh dumpxml GUEST_ID | grep -i tap |
| Get the CPU Pinning Settings for a Guest | virsh vcpupin GUEST_ID |
| Discover CPU Scheduling Parameters | virsh schedinfo GUEST_ID |
| stats of virtualized domains | virt-top |
| check attached disk | virsh domblklist GUEST_ID --details |
| Detach Disk | virsh detach-disk GUEST_ID sdc --config --live |
| More Info | RedHat Docs And Virsh_Command_Reference |
| Name | Command |
|---|---|
| assign privileges | GRANT ALL PRIVILEGES ON *.* TO 'username'@'localhost' WITH GRANT OPTION; |
| show processlist | show processlist\G;" | egrep "db\:" | awk -F: '{ print $2 }' | sort | uniq -c |
| Check connected Threads | show status like "%Threads_connected%"; |
| MySQL Thread Connected status | select user, count(user) from information_schema.processlist group by user ; |
| Check log Info | show variables like 'log_warnings'; |
| MariaDB Cheatsheet | CheetSheet |
| Name | Command |
|---|---|
| To create 100M size, enter | fallocate -l 100M file.out |
| Stress Memory/CPU | stress-ng --cpu 18 -m 18 --vm-bytes 17000M -t 36000 |
| Stress Memory | stress-ng --cpu 2 --malloc 1 --malloc-bytes 150% |
| Top Process. | ps -eo pid,ppid,cmd,%cpu,%mem --sort-%cpu= |
| Name | Command |
|---|---|
| To see which commands made the most system calls | perf top -F 49 -e raw_syscalls:sys_enter --sort comm,dso --show-nr-samples |
| To see latencies longer than a specific duration | perf trace --duration 200 |
| system calls within a period of time | perf trace -p $PID -s |
| trace-cmd | More Info on https://opensource.com/ |
| Name | Command |
|---|---|
| check cpu core | lscpu -e |
| check Hardware details | dmidecode -t system |
| Show HaProxy metrics | echo "show stat" | nc -U /var/lib/haproxy/stats-1 |
| What if nc command is not installed | timeout 1 bash -c "</dev/tcp/IP_ADDRESS/80" && echo Port open. || echo Port closed |
| What if nc command is not installed, without timeout cmd | bash -c "</dev/tcp/IP_ADDRESS/80" && echo Port open. || echo Port closed |
| Check remote SSL cert validity | openssl s_client -showcerts -connect cloud-svldev-1-storage.cisco.com:443 < /dev/null|openssl x509 -noout -text 2>/dev/null | grep After |
| Check local SSL cert validity | cat kubelet.crt | openssl x509 -noout -enddate |
| netstat connections sorted according to their state | netstat -ant | grep :$PORT | awk '{print $6}' | sort | uniq -c | sort -n |
| netstat check local connections | netstat -anp | grep -c $(hostname -i):5672 |
| Check iptables NAT rules | iptables -L -v -n |
| Find Large Files in Linux | du -a /dir/ | sort -n -r | head -n 20 |
| silent curl | curl -s https://openstack.com/v2/nova |