Skip to content

Commit b3935fc

Browse files
ryanaslettryanaslett
authored
Update MANUAL_STEPS.md (#4055)
Update the manual steps to prepare a smartos host. Some of these steps may be doable in ansible at some point.
1 parent 400b027 commit b3935fc

File tree

1 file changed

+114
-1
lines changed

1 file changed

+114
-1
lines changed

‎ansible/MANUAL_STEPS.md

+114-1
Original file line numberDiff line numberDiff line change
@@ -718,7 +718,120 @@ Note that while this is being done across all Docker hosts, you should disable [
718718

719719
## SmartOS
720720

721-
Joyent SmartOS machines use `libsmartsshd.so` for PAM SSH authentication in order to look up SSH keys allowed to access machines. Part of our Ansible setup removes this so we can only rely on traditional SSH authentication. Therefore, it is critcal to put `nodejs_test_*` public keys into `$USER/.ssh/authorized_keys` as appropriate or access will be lost and not recoverable after reboot or sshd restart (part of Ansible setup).
721+
The SmartOS machines are hosted by MNX.io. They are individual machines with the actual jenkins worker housed inside a VM on the host. To provision a new SmartOS jenkins host for testing we must do the following manual steps to prepare:
722+
1. Provision the host at MNX.io. The credentials for the mnx.io account are located in the admin logins file in the secrets repo.
723+
2. Configuration of the host environment
724+
3. Create the VM environment
725+
4. Configure the VM environment
726+
5. Create the Jenkins nodes/Open the jenkins firewall
727+
6. Ansible the VM
728+
729+
### Provisioning the Machines
730+
The host environment that houses the virtual machines currently relies on some older system libraries in order for compilation to succeed on smartos VM's. MNX.io has provided a base platform that has these older libs that we can use to make the workers.
731+
732+
1. Login to MNX.io.
733+
2. Select "Compute" and then "Custom Images" From the left sidebar
734+
3. Click on "Create Instance" on the `smartos-retro-20220407T001427Z`row.
735+
4. Then "click on the "Compute" option and pick c1.xlarge-ojsf (16GB Rab, 4vcpus, 200GB Disk), and then "Next"
736+
5. Name the instance test-mnx-smartosXX-x64-Y where XX is the version of smartos you plan on provisioning, and Y is an incremented number of similar instances
737+
6. Select MNX-Triton-Public (public) as the Network
738+
7. Add a tag of "role" = "test"
739+
8. Click Launch.
740+
741+
Instances should launch and be ready relatively quickly (less than 5 minutes)
742+
743+
### Configuring the Host Environment
744+
Once the instance is up you should be able to find its host IP address (choose Instances from the left column).
745+
Then, ssh to get into the host hypervisor: `ssh root@<IP ADDRESS> -i ~/.ssh/nodejs_build_test` (the nodejs_build_test key should be on the machine)
746+
The older images are still configured to point at a defunct joyent image archve. We have to point to the new mnx one:
747+
1. `mkdir /var/imgadm`
748+
2. `vi /var/imgadm/imgadm.conf`
749+
3. Put the following json into the contents:
750+
```
751+
{
752+
"dockerImportSkipUuids": true,
753+
"upgradedToVer": "3.0.0",
754+
"source": "https://images.mnx.io",
755+
"sources": [
756+
{
757+
"type": "imgapi",
758+
"url": "https://images.smartos.org"
759+
}
760+
]
761+
}
762+
```
763+
4. Find the UUID of the image for the version of smartos you want to run on the VM: `imgadm avail name=base-64-lts`
764+
```
765+
UUID NAME VERSION OS TYPE PUB
766+
[... older version omitted ...]
767+
1d05e788-5409-11eb-b12f-037bd7fee4ee base-64-lts 20.4.0 smartos zone-dataset 2021-01-11
768+
c8715b60-7e98-11ec-82d1-03d16599f529 base-64-lts 21.4.0 smartos zone-dataset 2022-01-26
769+
85d0f826-0131-11ed-973d-2bfeef68011c base-64-lts 21.4.1 smartos zone-dataset 2022-07-11
770+
93bdf06a-01ef-11ed-81ff-bf0efad842c7 base-64-lts 20.4.1 smartos zone-dataset 2022-07-12
771+
e44ed3e0-910b-11ed-a5d4-00151714048c base-64-lts 22.4.0 smartos zone-dataset 2023-01-10
772+
8adac45a-aca7-11ee-b53e-00151714048c base-64-lts 23.4.0 smartos zone-dataset 2024-01-06
773+
```
774+
5. Import the image
775+
smartos22: `imgadm import e44ed3e0-910b-11ed-a5d4-00151714048c`
776+
smartos23: `imgadm import 8adac45a-aca7-11ee-b53e-00151714048c`
777+
6. create new image_properties.json to define the VM we're creating:
778+
```
779+
{
780+
"brand": "joyent",
781+
"resolvers": [
782+
"8.8.8.8",
783+
"8.8.4.4"
784+
],
785+
"ram": 15360,
786+
"alias": "os1",
787+
"customer_metadata": {
788+
"root_authorized_keys": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQ+xhL3A6vERZoUUoQspAuydcEghmjiC0m8yETHqghnPr2Y5nfFjnzmNB8EPM5+/jsSwPBF0jUPFpAgWYYXAGQZ62hsovfVMcwqXlMYJyA+L/uGDX7KxtLhn6FcftxJgyHwbggC1kXuzmrtzX/4oHSQi9mth3sOuf4KxS2pE0nNqIy4lIEyfkutuIZa3dhKTYLVCklNCH+UjYBtVgIjvqBBoEKNcBNO4fhLM5MCS6/MpbkhTkTJBN/kvJYBfZ9xAh+2/gQc0ndtK+rJsOJaQ1yFnIHoJBTRZN3qS5O8aE7Tdtxh5G1B+S2BNujdPestpBHPGONsVtFgdMkRFfmp3lVCwoBOpNHL5zzkGHNuq2ViphxRsX5A9SgL4MP/5P3xKgxWbzYYEl/0ef612d8fQQmiMitX7STM5R/9aau1d3PUV9ZJLgNgIWLCjdifGgDJeO32fISGK/mHkCvR5zjhtd7wUBPKspOxbVNCj2C5mHGKhT3PlQNt1drwbLJUYndXCQN0EJuIeM81jcKGO6J51kOHco/NM3vQBGdn24efbj/R1gPtZsGmhc6ho/xUNEoSXY7sg6Ma++iYJ+nJqgUvvVPzpJxurcwABTjwqjFj9NZrUQtVchnONsfrpcEnZhOhUZSxhQquaWwpYfWw/slMe/B7BGRiRLY5payqlZkxWCUJw== test@build.nodejs.org",
789+
"user-script" : "/usr/sbin/mdata-get root_authorized_keys > ~root/.ssh/authorized_keys ; /usr/sbin/mdata-get root_authorized_keys > ~admin/.ssh/authorized_keys"
790+
},
791+
"nics": [
792+
{
793+
"interface": "net0",
794+
"nic_tag": "vswitch0",
795+
"gateway": "172.16.9.1",
796+
"gateways": [
797+
"172.16.9.1"
798+
],
799+
"netmask": "255.255.255.0",
800+
"ip": "172.16.9.3",
801+
"ips": [
802+
"172.16.9.3/24"
803+
],
804+
"primary": true
805+
}
806+
],
807+
"image_uuid": "8adac45a-aca7-11ee-b53e-00151714048c",
808+
"quota": 160
809+
}
810+
```
811+
7. create the vm with `vmadm create -f image_properties.json`
812+
8. ssh to the internal vm, proxying through the host: ```ssh root@172.16.9.3 -oProxyCommand="ssh root@192.207.255.126 -i ~/.ssh/nodejs_build_test -W %h:%p" -o StrictHostKeyChecking=no -i ~/.ssh/nodejs_build_test``` Note that the 192.207.255.126 is the IP address asssigned to the instance at MNX.(see image)
813+
9. install `htop` and `python` (use `pkgin search` to find the latest version of python to install based on the smartos version `python311` on smartos22, `python312` on smartos23)
814+
10. `pkgin install python311`
815+
11. Smartos22 extra steps:
816+
1. `pkg_alternatives manual python311`
817+
2. `pkgin install py311-expat-3.11.1nb1`
818+
3. `pkgin install openjdk17-17.0.9`
819+
4. `pkg_alternatives manual openjdk17-17.0.9`
820+
16. install pip: `python -m ensurepip --upgrade` and `python -m pip install packaging`
821+
17. Add the machine to the inventory with the proxycommand:
822+
```
823+
smartos23-x64-4:
824+
ip: 172.16.9.3
825+
ansible_ssh_common_args: '-o ProxyCommand="ssh -i ~/.ssh/nodejs_build_test -W %h:%p root@192.207.255.124"'
826+
ansible_user: root
827+
```
828+
18. Add the node to jenkins
829+
19. add the jenkins secret for the new node to the secrets repo
830+
20. Provision with ansible: `ansible-playbook ansible/playbooks/jenkins/worker/create.yml --limit "<HOSTNAME_TO_PROVISION>" -vv
831+
21. Ensure host can connect by modifying iptables on jenkins ci director.
832+
833+
834+
SmartOS machines use `libsmartsshd.so` for PAM SSH authentication in order to look up SSH keys allowed to access machines. Part of our Ansible setup removes this so we can only rely on traditional SSH authentication. Therefore, it is critcal to put `nodejs_test_*` public keys into `$USER/.ssh/authorized_keys` as appropriate or access will be lost and not recoverable after reboot or sshd restart (part of Ansible setup).
722835

723836
## IBM i
724837

0 commit comments

Comments
 (0)