Step by step: Creating RoboMaker Robot Resources using the CLI
Published on January 16, 2020awsdevopsrobomaker
RoboMaker is a tool that was introduced by AWS in 2018. It provides an easier way to simulate, build, and deploy robots that use Robot Operating System (ROS). Previously, I had been able to set up a convoluted system using AWS IoT Core for managing robots and deployments, which had worked just okay. When RoboMaker was released, I was excited to see if would be a match for what I needed. It was not. It did what it advertised very well, but any little change became very complicated to integrate into the RoboMaker flow, since you mostly did it all through the GUI. For example, when you create a deployment in RoboMaker, it automatically starts the update. This is great if your robot doesn't have a user interface or customers who want control over their updates. I wanted to be able to see when a new update is available and click a button to perform it. Another issue was that in order to use the robot application tool in Cloud9, you have to be okay with running your application on Docker. Due to the need to make kernel modifications, I wasn't able to use that portion of RoboMaker.
After a few months, I decided to give RoboMaker another shot, since it really did have a lot of the deployment and fleet features that we needed. We were able to get everything working from the RoboMaker GUI. As devops, we want to automate everything and using the GUI for creating new robots wasn't practical, especially with our modifications (automatic fleet assignment, group assignments, etc). The RoboMaker CLI docs are fairly well documented - but the process of creating resources in RoboMaker so that it actually works when using the CLI is nowhere to be found.
After watching the devtools network tab in RoboMaker while making a new robot, reading a lot of Greengrass, RoboMaker, and IoT docs, I was able to make a script that automates the setup of a new robot. I have some work to do (for example, making the rollback commands into a function so the code isn't repeated), but I wanted to get it out there so others could contribute. Let me know what you think!
Provisioning Variables
First, you'll want to define a few things:
echo "Enter the robot's serial number (ex. FX250123)"
read ROBOT_SN
if [ -z $ROBOT_SN ]
then
echo "A robot serial number is required."
exit
fi
THING_NAME=$ROBOT_SN-thing # thing name is used in AWS IoT
THING_GROUP=$ROBOT_SN-group # for greegrass group
THING_POLICY=$ROBOT_SN-policy # for greengrass policy
THING_CORE=$ROBOT_SN-core # for greengrass core
echo "Enter the customer's name (if no customer, leave blank)."
read CUSTOMER
if [ -z $CUSTOMER ]
then
CUSTOMER=Discovery
fi
TODAY=$(date '+%Y-%m-%d') # used as a thing attribute (optional)
ARCH=X86_64 # architecture of robot (can be ARMHF, ARM64, or X86_64)
ROLE_ARN=<your_role_arn> # the deployment role, assumes its already created
FLEET_ARN=<your_fleet_arn> # the fleet ARN that you want your robot to be initialized into
IOT_GROUP_NAME=<first_group_name>
IOT_GROUP_ARN=<name_of_first_group>
There are two inputs from the developer: ROBOT_SN and CUSTOMER. ROBOT_SN refers to the robot's serial number - it's unique identifier. We use this as the thing name, the robot name, and the greengrass prefix for all of the resources. The THING_NAME is the serial number plus -thing, which becomes the name of the AWS IoT Thing that is generated. THING_GROUP, THING_POLICY, and THING_CORE are the names used in generating the Greengrass resources. CUSTOMER, and DATE are used as attributes on the Thing that is created in IoT. ARCH is the architecture of the robot. There are three options: ARMHF, ARM64, or X86_64.
The ROLE_ARN is the ARN of the deployment role that Greengrass needs in order to interact with RoboMaker. This script assumes that it's already been created, but if you haven't done this as part of the RoboMaker tutorial or setup steps, follow it here: create-robot-role. I want all of my new robots to be put into a NewRobots fleet that I created, and the FLEET_ARN is the ARN of this initial fleet.
IOT_GROUP_NAME and IOT_GROUP_ARN refer to the initial group that the robots will join. This isn't necessary but it's helpful because you can assign additional policies to the group.
Environment
I run this script on my dev computer for each new robot. Sometimes I create a few at a time. Because of this, I wanted to be able to have a separate folder for all of the configs I make.
# ----------------------------------------------------------------------------------------
# Setup Environment
# This makes an output for your greengrass certs/config that go on the robot. PRefixed by the
# robot name so that you can make multiple robots at once.
# ----------------------------------------------------------------------------------------
mkdir -p robots/$ROBOT_SN/certs
mkdir -p robots/$ROBOT_SN/config
Create IoT Resources
AWS RoboMaker just uses a combo of IoT and Greengrass services under the hood. Next step is to provision these.
# ----------------------------------------------------------------------------------------
# IoT: createKeysAndCertificate
# returns: certificateArn, certificateId, certificatePem, keyPair (PublicKey, PrivateKey)
# ----------------------------------------------------------------------------------------
CERTS_RESPONSE=$(aws iot create-keys-and-certificate \
--set-as-active \
--certificate-pem-outfile robots/$ROBOT_SN/certs/$ROBOT_SN.cert.pem \
--public-key-outfile robots/$ROBOT_SN/certs/$ROBOT_SN.public.key \
--private-key-outfile robots/$ROBOT_SN/certs/$ROBOT_SN.private.key | jq -r '.')
CERT_ARN=$(echo $CERTS_RESPONSE | jq -r '.certificateArn')
CERT_ID=$(echo $CERTS_RESPONSE | jq -r '.certificateId')
if [ ! -z $CERT_ID ]
then
echo "A new set of certs was created with ID $CERT_ID"
else
echo "Certs could not be created."
exit
fi
Download the root.ca.pem. This is the Amazon Root CA and will be shared across all robots.
# ----------------------------------------------------------------------------------------
# Download the root CA
# ----------------------------------------------------------------------------------------
wget -O robots/$ROBOT_SN/certs/root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem
Create the IoT Thing and Attach the Cert to it
# ----------------------------------------------------------------------------------------
# IoT: createThing
# returns: thingName, thingArn, thingId
# ----------------------------------------------------------------------------------------
THING_ARN=$(aws iot create-thing \
--thing-name $THING_NAME \
--thing-type-name FX250_Chassis \
--attribute-payload "{\"attributes\": {\"customer\":\"$CUSTOMER\", \"date_provisioned\":\"$TODAY\", \"softwareVersion\":\"0.0.0\"}}" | jq -r '.thingArn')
if [ ! -z $THING_ARN ]
then
echo "A new thing was created for $THING_NAME"
else
echo "Thing thing could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot delete-certificate --certificate-id $CERT_ID
exit
fi
# ----------------------------------------------------------------------------------------
# IoT: attachThingPrincipal
# returns: {}
# ----------------------------------------------------------------------------------------
aws iot attach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
Create an IoT Policy and attach it to IoT Thing
# ----------------------------------------------------------------------------------------
# IoT: createPolicy (name: <robo_name>-policy)
# returns: policyArn, policyDocument, policyName, policyVersionId
# ----------------------------------------------------------------------------------------
POLICY_DETAILS=$(aws iot create-policy \
--policy-name $THING_POLICY \
--policy-document "{\"Version\": \"2012-10-17\",\"Statement\": [{\"Effect\": \"Allow\",\"Action\": [\"iot:Publish\",\"iot:Subscribe\",\"iot:Connect\",\"iot:Receive\"],\"Resource\": [\"*\"]},{\"Effect\": \"Allow\",\"Action\": [\"iot:GetThingShadow\",\"iot:UpdateThingShadow\",\"iot:DeleteThingShadow\"],\"Resource\": [\"*\"]},{\"Effect\": \"Allow\",\"Action\": [\"greengrass:*\"],\"Resource\": [\"*\"]}]}" | jq -r '.')
POLICY_ARN=$(echo $POLICY_DETAILS | jq -r '.policyArn')
if [ ! -z $POLICY_ARN ]
then
echo "A new policy called $THING_POLICY was created."
else
echo "Policy could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws iot delete-policy --policy-name $THING_POLICY
exit
fi
# ----------------------------------------------------------------------------------------
# # IoT: attachPolicy
# # returns: nothing
# ----------------------------------------------------------------------------------------
aws iot attach-policy --policy-name $THING_POLICY --target $CERT_ARN
Move the new IoT Thing into a Group
# ----------------------------------------------------------------------------------------
# IoT: add-thing-to-thing-group
# Returns: none
# ----------------------------------------------------------------------------------------
aws add-thing-to-thing-group \
--thing-group-name $IOT_GROUP_NAME \
--thing-group-arn $IOT_GROUP_ARN \
--thing-name $THING_NAME \
--thing-arn $THING_ARN \
--override-dynamic-groups
Create the Greengrass Resources
# ----------------------------------------------------------------------------------------
# Greengrass: createGroup
# ----------------------------------------------------------------------------------------
GG_GRP_ID=$(aws greengrass create-group --name $THING_GROUP| jq -r '.Id')
if [ ! -z $GG_GRP_ID ]
then
echo "The group ID is $GG_GRP_ID"
else
echo "The group ID wasn't set."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot delete-certificate --certificate-id $CERT_ID
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-thing --thing-name $THING_NAME
exit
fi
# ----------------------------------------------------------------------------------------
# Greengrass: associateRole to group
# assign role so group can access robomaker
# ----------------------------------------------------------------------------------------
RESPONSE=$(aws greengrass associate-role-to-group --group-id $GG_GRP_ID --role-arn $ROLE_ARN | jq --raw-output '.AssociatedAt')
if [ ! -z $RESPONSE ]
then
echo "The group was associated with the role at $RESPONSE"
else
echo "The group could not be associated with this role. Rolling back."
aws greengrass delete-role --group-id $GG_GRP_ID
exit
fi
# ----------------------------------------------------------------------------------------
# Greengrass: createCoreDefinition
# returns Name, Id, Arn, CreationTimestamp, LastUpdatedTimestamp
# ----------------------------------------------------------------------------------------
CORE_ID=$(aws greengrass create-core-definition --name $THING_CORE | jq -r '.Id')
if [ ! -z $CORE_ID ]
then
echo "The core was created with core ID $CORE_ID"
else
echo "The core could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws greengrass delete-group --group-id $GG_GRP_ID
aws greengrass delete-role --group-id $GG_GRP_ID
exit
fi
# ----------------------------------------------------------------------------------------
# Greengrass: createCoreDefinitionVersion
# returns Arn, Version, CreationTimestamp, Id
# ----------------------------------------------------------------------------------------
CORE_DEFINITION_VERSION_ARN=$(aws greengrass create-core-definition-version \
--core-definition-id $CORE_ID \
--cores CertificateArn=$CERT_ARN,Id=$CORE_ID,SyncShadow=true,ThingArn=$THING_ARN | jq -r '.Arn')
if [ ! -z $CORE_DEFINITION_VERSION_ARN ]
then
echo "The core definition version was created with ARN $CORE_DEFINITION_VERSION_ARN"
else
echo "The core definition version could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws greengrass delete-group --group-id $GG_GRP_ID
aws greengrass delete-role --group-id $GG_GRP_ID
aws greengrass delete-core-definition --core-definition-id $CORE_ID
exit
fi
# ----------------------------------------------------------------------------------------
# Greengrass: createGroupVersion
# returns Arn, Id, Version, CreationTimestamp
# ----------------------------------------------------------------------------------------
GROUP_VERSION_ARN=$(aws greengrass create-group-version \
--core-definition-version-arn $CORE_DEFINITION_VERSION_ARN \
--group-id $GG_GRP_ID | jq -r '.Arn')
if [ ! -z $GROUP_VERSION_ARN ]
then
echo "The new group version was created: $GROUP_VERSION_ARN"
else
echo "The new group version could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws greengrass disassociate-role-from-group --group-id $GG_GRP_ID
aws greengrass delete-group --group-id $GG_GRP_ID
aws greengrass delete-core-definition --core-definition-id $CORE_ID
exit
fi
Create Robomaker Robot
# ----------------------------------------------------------------------------------------
# RoboMaker: create robot
# ----------------------------------------------------------------------------------------
ROBOT_DETAILS=$(aws robomaker create-robot --name $ROBOT_SN --architecture $ARCH --greengrass-group-id $GG_GRP_ID | jq -r '.')
ROBOT_ARN=$(echo $ROBOT_DETAILS | jq -r '.arn')
if [ ! -z $ROBOT_ARN ]
then
echo "The robot with ARN $ROBOT_ARN was successfully created."
else
echo "The robot could not be created. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws greengrass disassociate-role-from-group --group-id $GG_GRP_ID
aws greengrass delete-group --group-id $GG_GRP_ID
aws greengrass delete-core-definition --core-definition-id $CORE_ID
exit
fi
# ----------------------------------------------------------------------------------------
# Robomaker: assign robot to new fleet
# Assumes the fleet has already been created (I register all new ones to a new robots fleet)
# ----------------------------------------------------------------------------------------
FLEET_DETAILS=$(aws robomaker register-robot --fleet $FLEET_ARN --robot $ROBOT_ARN | jq -r '.')
FLEET_ARN=$(echo $FLEET_DETAILS | jq -r '.fleet')
if [ ! -z $FLEET_ARN ]
then
echo "The robot was successfully associated with the fleet $FLEET_ARN"
else
echo "The robot could not be associated with the fleet. Rolling back."
aws iot detach-thing-principal --thing-name $THING_NAME --principal $CERT_ARN
aws iot update-certificate --certificate-id $CERT_ID --new-status INACTIVE
aws iot detach-policy --policy-name $THING_POLICY --target $CERT_ARN
aws iot delete-policy --policy-name $THING_POLICY
aws iot delete-certificate --certificate-id $CERT_ID
aws iot delete-thing --thing-name $THING_NAME
aws greengrass disassociate-role-from-group --group-id $GG_GRP_ID
aws greengrass delete-group --group-id $GG_GRP_ID
aws greengrass delete-core-definition --core-definition-id $CORE_ID
aws robomaker delete-robot --robot $ROBOT_ARN
exit
fi
Create Greengrass config to go on Physical Robot
# ----------------------------------------------------------------------------------------
# IoT: describe-endpoint
# response: endpointAddress
# ----------------------------------------------------------------------------------------
ENDPOINT_ADDR=$(aws iot describe-endpoint --endpoint-type iot:Data-ATS | jq -r '.endpointAddress')
# ----------------------------------------------------------------------------------------
# Create config and save to robot folder
# ----------------------------------------------------------------------------------------
echo "{\"coreThing\":{\"caPath\":\"root.ca.pem\",\"certPath\":\"$ROBOT_SN.cert.pem\",\"ggHost\":\"greengrass-ats.iot.us-east-1.amazonaws.com\",\"iotHost\":\"$ENDPOINT_ADDR\",\"keepAlive\":600,\"keyPath\":\"$ROBOT_SN.private.key\",\"thingArn\":\"$THING_ARN\"},\"crypto\":{\"caPath\":\"file:///greengrass/certs/root.ca.pem\",\"principals\":{\"IoTCertificate\":{\"CertificatePath\":\"file:///greengrass/certs/$ROBOT_SN.cert.pem\",\"PrivateKeyPath\":\"file:///greengrass/certs/$ROBOT_SN.private.key\"},\"SecretsManager\":{\"PrivateKeyPath\":\"file:///greengrass/certs/$ROBOT_SN.private.key\"}}},\"managedRespawn\":false,\"runtime\":{\"allowFunctionsToRunAsRoot\":\"yes\",\"cgroup\":{\"useSystemd\":\"yes\"}}}" > robots/$ROBOT_SN/config/config.json
After completing all of those steps, you'll have provisioned a AWS RoboMaker Robot, including its fleet, all associated IoT and Greengrass resources, and the certificates and config.json that gets put on the physical robot, along with the core Greengrass software. You can find the full script here. Thanks for reading!