mod_rest

Introduction

This is yet another RESTful API for sending and receiving stanzas via Prosody. It can be used to build bots and components implemented as HTTP services. It is the spiritual successor to mod_post_msg and absorbs use cases from mod_http_rest and mod_component_http and other such modules floating around the Internet.

Usage

You make a choice: install via VirtualHosts or as a Component. User authentication can be used when installed via VirtualHost, and OAuth2 can be used for either.

On VirtualHosts

This enables rest on the VirtualHost domain, enabling user authentication to secure the endpoint. Make sure that the modules_enabled section is immediately below the VirtualHost entry so that it’s not under any Component sections. EG:

VirtualHost "chat.example.com"
modules_enabled = {"rest"}

As a Component

If you install this as a component, you won’t be able to use user authentication above, and must use OAuth2 authentication outlined below.

Component "chat.example.com" "rest"
component_secret = "dmVyeSBzZWNyZXQgdG9rZW4K"
modules_enabled = {"http_oauth2"}

User authentication

To enable user authentication, edit the “admins = { }” section in prosody.cfg.lua, EG:

admins = { "admin@chat.example.com" }

To set up the admin user account:

prosodyctl adduser admin@chat.example.com

and lastly, drop the “@host” from the username in your http queries, EG:

curl \
  https://chat.example.com:5281/rest/version/chat.example.com \
  -k \
  --user admin \
  -H 'Accept: application/json'

OAuth2

mod_http_oauth2 can be used to grant bearer tokens which are accepted by mod_rest. Tokens can be passed to curl like --oauth2-bearer dmVyeSBzZWNyZXQgdG9rZW4K instead of using --user.

Sending stanzas

The API endpoint becomes available at the path /rest, so the full URL will be something like https://conference.chat.example.com:5281/rest.

To try it, simply curl an XML stanza payload:

curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/xmpp+xml' \
    --data-binary '<message type="chat" to="user@example.org">
            <body>Hello!</body>
        </message>'

or a JSON payload:

curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/json' \
    --data-binary '{
           "body" : "Hello!",
           "kind" : "message",
           "to" : "user@example.org",
           "type" : "chat"
        }'

The Content-Type header is important!

Parameters in path

New alternative format with the parameters kind, type, and to embedded in the path:

curl https://prosody.example:5281/rest/message/chat/john@example.com \
    --user username \
    -H 'Content-Type: text/plain' \
    --data-binary 'Hello John!'

Replies

A POST containing an <iq> stanza automatically wait for the reply, long-polling style.

curl https://prosody.example:5281/rest \
    --user username \
    -H 'Content-Type: application/xmpp+xml' \
    --data-binary '<iq type="get" to="example.net">
            <ping xmlns="urn:xmpp:ping"/>
        </iq>'

Replies to other kinds of stanzas that are generated by the same Prosody instance MAY be returned in the HTTP response. Replies from other entities (connected clients or remote servers) will not be returned, but can be forwarded via the callback API described in the next section.

Simple info queries

A subset of IQ stanzas can be sent as simple GET requests

curl https://prosody.example:5281/rest/version/example.com \
    --user username \
    -H 'Accept: application/json'

The supported queries are

archive
disco
extdisco
items
lastactivity
oob
payload
ping
stats
version

Receiving stanzas

TL;DR: Set this webhook callback URL, get XML POST-ed there.

Component "rest.example.net" "rest"
rest_callback_url = "http://my-api.example:9999/stanzas"

The callback URL supports a few variables from the stanza being sent, namely {kind} (e.g. message, presence, iq or meta) and ones corresponding to stanza attributes: {type}, {to} and {from}.

The preferred format can be indicated via the Accept header in response to an OPTIONS probe that mod_rest does on startup, or by configuring:

rest_callback_content_type = "application/json"

Example callback looks like:

POST /stanzas HTTP/1.1
Content-Type: application/xmpp+xml
Content-Length: 102

<message to="bot@rest.example.net" from="user@example.com" type="chat">
<body>Hello</body>
</message>

or as JSON:

POST /stanzas HTTP/1.1
Content-Type: application/json
Content-Length: 133

{
   "body" : "Hello",
   "from" : "user@example.com",
   "kind" : "message",
   "to" : "bot@rest.example.net",
   "type" : "chat"
}

Which stanzas

The set of stanzas routed to the callback is determined by these two settings:

rest_callback_stanzas: The stanza kinds to handle, defaults to { "message", "presence", "iq" }
rest_callback_events: For the selected stanza kinds, which events to handle. When loaded on a Component, this defaults to { "bare", "full", "host" }, while on a VirtualHost the default is { "host" }.

Events correspond to which form of address was used in the to attribute of the stanza.

bare: localpart@hostpart
full: localpart@hostpart/resourcepart
host: hostpart

The following example would handle only stanzas like <message to="anything@hello.example"/>

Component "hello.example" "rest"
rest_callback_url = "http://hello.internal.example:9003/api"
rest_callback_stanzas = { "message" }
rest_callback_events = { "bare" }

Replying

To accept the stanza without returning a reply, respond with HTTP status code 202 or 204.

HTTP status codes in the 4xx and 5xx range are mapped to an appropriate stanza error.

For full control over the response, set the Content-Type header to application/xmpp+xml and return an XMPP stanza as an XML snippet.

HTTP/1.1 200 Ok
Content-Type: application/xmpp+xml

<message type="chat">
<body>Yes, this is bot</body>
</message>

Payload format

JSON

{
   "body" : "Hello!",
   "kind" : "message",
   "type" : "chat"
}

Further JSON object keys as follows:

Messages

kind: "message"
type: Commonly "chat" for 1-to-1 messages and "groupchat" for group chat messages. Others include "normal", "headline" and "error".
body: Human-readable message text.
subject: Message subject or MUC topic.
html: HTML.
oob_url: URL of an out-of-band resource, often used for images.

Presence

kind: "presence"
type: Empty for online or "unavailable" for offline.
show: Online status, away, dnd etc.
status: Human-readable status message.

Info-Queries

Only one type of payload can be included in an iq.

kind: "iq"
type: "get" or "set" for queries, "response" or "error" for replies.
ping: Send a ping. Get a pong. Maybe.
disco: Retrieve service discovery information about an entity.
items: Discover list of items (other services, groupchats etc).

XML

<message type="" id="" to="" from="" xml:lang="">
...
</message>

An XML declaration (<?xml?>) MUST NOT be included.

The payload MUST contain one (1) message, presence or iq stanza.

The stanzas MUST NOT have an xmlns attribute, and the default/empty namespace is treated as jabber:client.

Examples

Python / Flask

Simple echo bot that responds to messages as XML:

from flask import Flask, Response, request
import xml.etree.ElementTree as ET

app = Flask("echobot")


@app.before_request
def parse():
    request.stanza = ET.fromstring(request.data)


@app.route("/", methods=["POST"])
def hello():
    if request.stanza.tag == "message":
        return Response(
            "<message><body>Yes this is bot</body></message>",
            content_type="application/xmpp+xml",
        )

    return Response(status=501)


if __name__ == "__main__":
    app.run()

And a JSON variant:

from flask import Flask, Response, request, jsonify

app = Flask("echobot")


@app.route("/", methods=["POST"])
def hello():
    print(request.data)
    if request.is_json:
        data = request.get_json()
        if data["kind"] == "message":
            return jsonify({"body": "hello"})

    return Response(status=501)


if __name__ == "__main__":
    app.run()

Remember to set rest_callback_content_type = "application/json" for this to work.

JSON mapping

This section describes the JSON mapping. It can’t represent any possible stanza, for full flexibility use the XML mode.

Stanza basics

kind: String representing the kind of stanza, one of "message", "presence" or "iq".
type: String with the type of stanza, appropriate values vary depending on kind, see RFC 6121. E.g."chat" for message stanzas etc.
to: String containing the XMPP Address of the destination / recipient of the stanza.
from: String containing the XMPP Address of the sender the stanza.
id: String with a reasonably unique identifier for the stanza.

Basic Payloads

Messages

body: String, human readable text message.
subject: String, human readable summary equivalent to an email subject or the chat room topic in a type:groupchat message.

Presence

show: String representing availability, e.g. "away", "dnd". No value means a normal online status. See RFC 6121 for the full list.
status: String with a human readable text message describing availability.

More payloads

Messages

state: String with current chat state, e.g. "active" (default) and "composing" (typing).
html: String with HTML allowing rich formatting. MUST be contained in a <body> element.
oob_url: String with an URL of an external resource.

Presence

muc: Object with MUC related properties.

IQ

ping: Boolean, a simple ping query. “Pongs” have only basic fields presents.
version: Map with name, version fields, and optionally an os field, to describe the software.

Service Discovery

disco

Boolean true in a kind:iq type:get for a service discovery query.

Responses have a map containing an array of available features in the features key and an array of “identities” in the identities key. Each identity has a category and type field as well as an optional name field. See XEP-0030 for further details.

items

Boolean true in a kind:iq type:get for a service discovery items list query. The response contain an array of items like {"jid":"xmpp.address.here","name":"Description of item"}.

extensions

Map of extended feature discovery (see XEP-0128) data with FORM_DATA fields as the keys pointing at maps with the rest of the data.

Ad-Hoc Commands

Used to execute arbitrary commands on supporting entities.

command

String representing the command node or Map with the following possible fields:

node: Required string with node from disco#items query for the command to execute.
action: Optional enum string defaulting to "execute". Multi-step commands may involve "next", "prev", "complete" or "cancel".
actions: Set (map of strings to true) with available actions to proceed with in multi-step commands.
status: String describing the status of the command, normally "executing".
sessionid: Random session ID issued by the responder to identify the session in multi-step commands.
note: Map with "type" and "text" fields that carry simple result information.
form: Data form with description of expected input and data types in the next step of multi-step commands. TODO document format.
data: Map with only the data for result dataforms. Fields may be strings or arrays of strings.

Example

Discovering commands:

{
   "items" : {
      "node" : "http://jabber.org/protocol/commands"
   },
   "id" : "8iN9hwdAAcfTBchm",
   "kind" : "iq",
   "to" : "example.com",
   "type" : "get"
}

Response:

{
   "from" : "example.com",
   "id" : "8iN9hwdAAcfTBchm",
   "items" : [
      {
         "jid" : "example.com",
         "name" : "Get uptime",
         "node" : "uptime"
      }
   ],
   "kind" : "iq",
   "type" : "result"
}

Execute the command:

{
   "command" : {
      "node" : "uptime"
   },
   "id" : "Jv-87nRaP6Mnrp8l",
   "kind" : "iq",
   "to" : "example.com",
   "type" : "set"
}

Executed:

{
   "command" : {
      "node" : "uptime",
      "note" : {
         "text" : "This server has been running for 0 days, 20 hours and 54 minutes (since Fri Feb  7 18:05:30 2020)",
         "type" : "info"
      },
      "sessionid" : "6380880a-93e9-4f13-8ee2-171927a40e67",
      "status" : "completed"
   },
   "from" : "example.com",
   "id" : "Jv-87nRaP6Mnrp8l",
   "kind" : "iq",
   "type" : "result"
}

TODO

Describe multi-step commands with dataforms.
Versioned API, i.e. /v1/stanzas
Bind resource to webhook/callback

Compatibility

Requires Prosody trunk / 0.12

Installation

With the plugin installer in Prosody 0.12 you can use:

sudo prosodyctl install --server=https://modules.prosody.im/rocks/ mod_rest

For earlier versions see the documentation for installing 3rd party modules

Latest changes

Add XEP-0191 blocklist retrieval to JSON mapping 2024-11-06
Add XEP-0377 spam reporting to JSON mapping 2024-11-05
Add XEP-0191 blocking to JSON mapping 2024-11-05
All community modules: Unify file extention of Markdown files to .md 2024-10-22
Wrap mod_tokenauth errors 2024-09-21

mod_rest

Introduction

Usage

On VirtualHosts

As a Component

User authentication

OAuth2

Sending stanzas

Parameters in path

Replies

Simple info queries

Receiving stanzas

Which stanzas

Replying

Payload format

JSON

Messages

Presence

Info-Queries

XML

Examples

Python / Flask

JSON mapping

Stanza basics

Basic Payloads

Messages

Presence

More payloads

Messages

Presence

IQ

Service Discovery

Ad-Hoc Commands

Example

TODO

Compatibility

Installation

Latest changes

Details

Authors

Links