ZPK Moderation API - Technical Documentation

Response structure

Example response to a success request.

In this example we can see two different messages after processing.

{
   "success": true,
   "messages": [
      {
         "input": {
            "text": "Estoy jugando al videojuego este, pero los enemigos no paran de matarme xd los matare a todos",
            "source_id": "test",
            "message_id": ""
         },
         "categories": {
            "hate": {
               "id": "hate",
               "score": "0.000185327",
               "danger": false
            },
            "hate_threatening": {
               "id": "hate_threatening",
               "score": "0.000000335",
               "danger": false
            },
            "self_harm": {
               "id": "self_harm",
               "score": "0.000000909",
               "danger": false
            },
            "sexual": {
               "id": "sexual",
               "score": "0.000002828",
               "danger": false
            },
            "sexual_minors": {
               "id": "sexual_minors",
               "score": "0.000000048",
               "danger": false
            },
            "violence": {
               "id": "violence",
               "score": "0.767623782",
               "danger": true
            },
            "violence_graphic": {
               "id": "violence_graphic",
               "score": "0.000000240",
               "danger": false
            }
         }
      },
      {
         "input": {
            "text": "Los matare a todos",
            "source_id": "test",
            "message_id": ""
         },
         "categories": {
            "hate": {
               "id": "hate",
               "score": "0.960755408",
               "danger": true
            },
            "hate_threatening": {
               "id": "hate_threatening",
               "score": "0.702908933",
               "danger": true
            },
            "self_harm": {
               "id": "self_harm",
               "score": "0.000000001",
               "danger": false
            },
            "sexual": {
               "id": "sexual",
               "score": "0.000002006",
               "danger": false
            },
            "sexual_minors": {
               "id": "sexual_minors",
               "score": "0.000000261",
               "danger": false
            },
            "violence": {
               "id": "violence",
               "score": "0.997496307",
               "danger": true
            },
            "violence_graphic": {
               "id": "violence_graphic",
               "score": "0.000001936",
               "danger": false
            }
         }
      }
   ],
   "cost": 0.003
}

Example of response with general error

Example with an incorrect request that will NOT be processed.

{
   "success": false,
   "errors": [
      {
         "id": "INVALID_PARAMETER",
         "message": "Messages array is empty.",
		 "on": "messages"
      }
	]
}
				

This API assigns each message a score and a danger alert.

Alerts are usually activated when the score approaches 0.7, however the AI can decide whether to trigger the alert or not depending on a contextualization analysis of the message.

For example: A message could contain text talking about violence and therefore have a relatively high violence.score, but the AI ​​could decide not to trigger a violence.danger alert if that message refers to a shooter video game and does not to a real threat.

In any case, it is you who must determine the threshold of each category that your application must tolerate, since usually the same restrictions will not apply in a community of adult audiences where all users are of legal age, or in a community suitable for all audiences.

The ZPK Moderation API will assign analyzes where a source_id has been specified to a specific source.

When a source is analyzed, or when the source status endpoint is queried, a total average score is returned that you can use to determine the 'overall' attitude of the sender of the messages.

A source can be any message source that your application determines, for example, a specific user, or a specific chat conversation.

This endpoint will return for a specific source, the current score associated with that source in each dangerous category.

It also returns information on the number of messages processed, and a 'global' attitude score.

Source stats endpoint

Source info, response structure

Source Info Request, example response

{
   "success": true,
   "source_id": "test",
   "scores": {
      "hate": {
         "average_score": 0.3245996,
         "percentage_total_messages": 29.63
      },
      "hate_threatening": {
         "average_score": 0.1642738,
         "percentage_total_messages": 29.63
      },
      "self_harm": {
         "average_score": 0.0004603,
         "percentage_total_messages": 0
      },
      "sexual": {
         "average_score": 0.0018926,
         "percentage_total_messages": 0
      },
      "sexual_minors": {
         "average_score": 0.0001772,
         "percentage_total_messages": 0
      },
      "violence": {
         "average_score": 0.5734031,
         "percentage_total_messages": 44.44
      },
      "violence_graphic": {
         "average_score": 0.0218495,
         "percentage_total_messages": 0
      }
   },
   "cost": 0.005
}

SDK in PHP to detect dangerous messages

ZPK-PHP is an open source library installable through composer that will allow you to integrate our APIs into your PHP project with minimal effort.

Composer install

composer require zpksystems/phpzpk

Example request to scan messages

<?php

$application = new zpkApplication('app_id','api_key');
$moderator = new zpkModerator($application);

$moderator->addText( ['text'=>'I want to k**l myself.'] );

echo json_encode($moderator->scan(),
 JSON_PRETTY_PRINT|JSON_UNESCAPED_UNICODE);