A few quick notes on getting the AWS IoT C SDK running on Linux.
I played a lot this past weekend with AWS IoT things. I finally decided that my friends advice was good: get something running on a known, reliable platform and then change variables once I understood what I was doing. So I literally went right back to basics: the IoT Embedded C SDK. I’m going to get that running, then I’ll go to the ESP32 port.
It Works, But Details Matter
AWS IoT has security built in, and the details matter. I just wanted to do the simple “mqtt_demo_mutual_auth” case. I spent some time getting things right. Now, I don’t want to say this is the ONLY way to do it… but it’s what I had to do.
Why Do This?
Turns out this is hugely valuable. If you do this, and get it working, you know that your AWS-side works. You can work it all out at the linux command line where logs are easy and the iterative process of development is trivial. Trying to do this AND deal with flashing an embedded controller is adding a lot of variables for things to go wrong. And makes it really hard to troubleshoot.
Once you have all this working, you can translate the SAME thing name and topics to your embedded code and at least know it was working on linux.
On the AWS Console
Make a Thing
Start with making a thing in the Console. Naviage to “AWS IoT –> Getting Started” and select “Onboard a device” by clicking “Get Started” and then click “Get started” again on the next screen. I selected “Linux/OSX” and Python, then clicked “Next.”
This seemed important: I named my thing “testclient” - discovered through trial and error.
Download the zip by clicking on “Linux/OSX” and save the file locally. In a local terminal, unzip that file to a folder and remember that location. You should have these files now:
connect_device_package.zip
start.sh
testclient.cert.pem
testclient.private.key
testclient.public.key
If you want to test the keys, follow the normal directions in the console.
Create a Policy That Matches the Demo
The policy that was created automatically above is good for the python test code that was in the zip file, but not for the C SDK. In the Console, navigate to Secure –> Policies and click “Create” on the upper right side. Name the policy “testclient-policy” and click “Advanced mode” to open an inline editor. Paste this policy, replacing $REGION:$ACCOUNT_NUMBER with the region string and account number for you.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iot:Connect",
"Resource": "arn:aws:iot:$REGION:$ACCOUNT_NUMBER:client/${iot:Connection.Thing.ThingName}"
},
{
"Effect": "Allow",
"Action": "iot:Subscribe",
"Resource": "arn:aws:iot:$REGION:$ACCOUNT_NUMBER:topicfilter/${iot:Connection.Thing.ThingName}/example/topic"
},
{
"Effect": "Allow",
"Action": "iot:Receive",
"Resource": "arn:aws:iot:$REGION:$ACCOUNT_NUMBER:topic/${iot:Connection.Thing.ThingName}/example/topic"
},
{
"Effect": "Allow",
"Action": "iot:Publish",
"Resource": "arn:aws:iot:$REGION:$ACCOUNT_NUMBER:topic/${iot:Connection.Thing.ThingName}/example/topic"
}
]
}
Click “Create” to make the policy.
Associate the Policy with the Certs
This is important. Miss this and you won’t be able to communicate over MQTT. Naviage to the Console Manage –> Things and select the ’testclient’ thing you created. You should see that it has a certificate attached. Click on that cert. You will be taken to that certicate configuration. Navigating to the cert in this way ensures that you have the right cert selected.
Click Actions in the right corner and select “Attach policy.” You will get a drop-down dialog box listing the policies you have created. Select the new “testclient-policy” that you just created above, and click “Attach.”
All this is IMPORTANT
The certs ensure that only authorized clients can connect, and that clients can trust that the endpoint really is AWS, and that the communications will be secure.
The policy limits what the client can do. Specifically, that policy allows any thing name to connect to the MQTT broker, subscribe, send and receive to/from any topic named like this:
$ThingName/example/topic
The client CANNOT use any other topics at all. This is a majority of the point of AWS IoT: to provide a common service while ensuring high security for all the clients.
On the Build PC (Linux only was tested)
Clone the SDK
To proceed to the SDK testing, clone the repo locally per the instructions.
git clone --recurse-submodules git@github.com:aws/aws-iot-device-sdk-embedded-C.git
cd aws-iot-device-sdk-embedded-C
Find your AWS Endpoint
The easiest way is to use the AWS CLI:
aws iot describe-endpoint --endpoint-type iot:Data-ATS
Configure the Demo
You will need to ensure you have cmake installed. Run this command, filling in your details. IMPORTANT: don’t use relative paths for the cert/key files. Use the fully qualified path.
Here’s my command:
cmake -S. -Bbuild -DAWS_IOT_ENDPOINT="a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com" -DCLIENT_CERT_PATH="/home/gherlein/certs/testclient.cert.pem" -DCLIENT_PRIVATE_KEY_PATH="/home/gherlein/certs/testclient.private.key"
Then, generate the makefile:
cmake -S. -Bbuild
If all went well, you should see this at the end of your output:
-- Configuring done
-- Generating done
-- Build files have been written to: /home/gherlein/src/aws/aws-iot-device-sdk-embedded-C/build
Now, build:
cd build
make -j8 ## pick a number less than 8 if you so choose
cd bin
You should now see something like this:
gherlein@mars:~/src/aws/aws-iot-device-sdk-embedded-C/build/bin$ ls
certificates http_demo_plaintext mqtt_demo_mutual_auth pkcs11_demo_management_and_rng pkcs11_demo_objects
http_demo_basic_tls jobs_demo_mosquitto ota_demo_core_http pkcs11_demo_mechanisms_and_digests pkcs11_demo_sign_and_verify
Run the Demo
Here’s how I ran it and what good results look like:
gherlein@mars:~/src/aws/aws-iot-device-sdk-embedded-C/build/bin$ ./mqtt_demo_mutual_auth
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:625] Establishing a TLS session to a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com:8883.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1325] Creating an MQTT connection to a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com.
[INFO] [MQTT] [core_mqtt.c:885] Packet received. ReceivedBytes=2.
[INFO] [MQTT] [core_mqtt_serializer.c:970] CONNACK session present bit not set.
[INFO] [MQTT] [core_mqtt_serializer.c:912] Connection accepted.
[INFO] [MQTT] [core_mqtt.c:1563] Received MQTT CONNACK successfully from broker.
[INFO] [MQTT] [core_mqtt.c:1829] MQTT connection established with the broker.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1096] MQTT connection successfully established with broker.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1358] A clean MQTT connection is established. Cleaning up all the stored outgoing publishes.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1375] Subscribing to the MQTT topic testclient/example/topic.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1158] SUBSCRIBE sent for topic testclient/example/topic to broker.
[INFO] [MQTT] [core_mqtt.c:885] Packet received. ReceivedBytes=3.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:980] Subscribed to the topic testclient/example/topic. with maximum QoS 1.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1419] Sending Publish to the MQTT topic testclient/example/topic.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1255] PUBLISH sent for topic testclient/example/topic to broker with packet ID 2.
[INFO] [MQTT] [core_mqtt.c:885] Packet received. ReceivedBytes=2.
[INFO] [MQTT] [core_mqtt.c:1161] Ack packet deserialized with result: MQTTSuccess.
[INFO] [MQTT] [core_mqtt.c:1174] State record updated. New state=MQTTPublishDone.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1007] PUBACK received for packet id 2.
And there you have it. A working demo, at least on linux.
Troubleshooting
If your certs are wrong, you will get output that looks like this:
gherlein@mars:~/src/aws/aws-iot-device-sdk-embedded-C/build/bin$ ./mqtt_demo_mutual_auth
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:625] Establishing a TLS session to a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com:8883.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:396] SSL_CTX_use_certificate_chain_file failed to import client certificate at ~/certs.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:627] Setting up credentials failed.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:672] Failed to establish a TLS connection.
Note the “Setting up credentials failed” entry. Go back and triple check that your certs are set up correctly.
If your policy is not right, or not attached to the certs, you will get output like this:
gherlein@mars:~/src/aws/aws-iot-device-sdk-embedded-C/build/bin$ ./mqtt_demo_mutual_auth
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:625] Establishing a TLS session to a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com:8883.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1325] Creating an MQTT connection to a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com.
[INFO] [MQTT] [core_mqtt.c:885] Packet received. ReceivedBytes=2.
[INFO] [MQTT] [core_mqtt_serializer.c:970] CONNACK session present bit not set.
[INFO] [MQTT] [core_mqtt_serializer.c:912] Connection accepted.
[INFO] [MQTT] [core_mqtt.c:1563] Received MQTT CONNACK successfully from broker.
[INFO] [MQTT] [core_mqtt.c:1829] MQTT connection established with the broker.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1096] MQTT connection successfully established with broker.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1358] A clean MQTT connection is established. Cleaning up all the stored outgoing publishes.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1375] Subscribing to the MQTT topic testclient/example/topic.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1158] SUBSCRIBE sent for topic testclient/example/topic to broker.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:760] Failed to receive data over network: SSL_read failed: ErrorStatus=EVP lib.
[ERROR] [MQTT] [core_mqtt_serializer.c:2394] A single byte was not read from the transport: transportStatus=-1.
[ERROR] [MQTT] [core_mqtt.c:1319] Receiving incoming packet length failed. Status=MQTTRecvFailed
[ERROR] [MQTT] [core_mqtt.c:2191] Exiting process loop due to failure: ErrorStatus=MQTTRecvFailed
[ERROR] [DEMO] [mqtt_demo_mutual_auth.c:1395] MQTT_ProcessLoop returned with status = MQTTRecvFailed.
[INFO] [DEMO] [mqtt_demo_mutual_auth.c:1476] Disconnecting the MQTT connection with a11cxor2mooi1v-ats.iot.us-west-1.amazonaws.com.
[INFO] [MQTT] [core_mqtt.c:2149] Disconnected from the broker.
This indicates that the client can connect, but cannot work on those topics on MQTT. Triple check your policy is correct, and that it is attached to that certificate.
