SMS file format
Text messages
An SMS file is a text file that contains the message and a header. You have to
store all SM you want to send in these files in the outgoing directory.
The filename does not matter but it has to be unique. You may use the
mktemp command to generate unique filenames.
Easy example:
To: 491721234567
Hello, this is the sms.
|
Write the phone number in international format without the leading +.
When you like to send a message to a short number (for example to
order a ringtone), then preceed it with an "s".
More complex example:
To: 491721234567
Flash: yes
Alphabet: ISO
Hello Stefan, how are you?
|
NOTE: Headers are case-sensitive.
You can add as many header lines, as you like. When the program finds an unknown header, it simply
ignores that line. You can use the following header lines:
Alphabet |
Tells the program what character set is used in this sms file. Possible values are;
binary |
The short message contains 8-bit binary data, no text. |
GSM |
7 bit character set, as described in the GSM specification. |
ISO Latin Ansi |
Normal 8 bit character set, also called Ansi or Latin-9. All three keywords do the same. |
UCS Chinese Unicode |
UCS2 character set, maximum 70 characters. All three values do the same. The header must be written with an 8 bit character set but the message text part must be written with the 16 bit Unicode (big endian) character set. Please checkout the scripts directory, it contains some useful scripts for file format conversion. |
UTF-8 |
8 bit multibyte character set. Available since version 3.1.16. |
The program checks only the first 3 characters of that line, therefore keywords like ISO-8859-15 or UCS-2 will also work fine.
NOTE the difference of ISO and UTF-8 since version 3.1.16:
Handling of alphabets is enhanced and all conversions are now done using internal routines
which work well with cyrillic languages too.
For backward compatibility, the alphabet of message file still defaults to ISO-8859-15,
and this can be changed to UTF-8 in the configuration, or a header Alphabet: UTF
can should be used.
Alphabet: ISO
-
depending on the global setting outgoing_utf8 (which defaults to yes), UTF-8 is still accepted and converted.
-
optical replacement is done, depending on the modem setting cs_convert_optical (which defaults to yes).
-
characters are converted to the GSM alphabet, and missing characters are dropped.
Alphabet: UTF
-
like with ISO, but if any characters cannot be converted to the GSM alphabet,
whole message is converted to the UCS2 alphabet.
-
if conversion to UCS2 is done, each message part will have less characters and
therefore more parts will be required, but the text delivers as it was written.
|
Autosplit |
Controls if and how the program splits large text messages. Without this line, the setting from config file is used.
0 | disabled |
1 | enabled, no part-number |
2 | enabled, text numbers |
3 | enabled, concatenated format (not supported by some phones) |
|
Class |
Number. Available since version 3.1.16.
Sets the Message Class, 0...3.
|
DCS_hex |
Value. Available since version 3.1.16.
Sets Data Coding Scheme in the PDU.
Note that value must be represented as two hexadecimal digits.
|
Flash |
Boolean value. If yes, then the message appears directly on the phones display. Most phones support this feature, but not all. |
From |
Senders name or phone number. This field has currently no function to the software. |
Hex |
Boolean value. Available since version 3.0.
Together with Alphabet: binary setting the binary data can be presented in hexadecimal format.
Since version 3.1.16 this can be used with text messages other than UCS2 too.
One byte should be presented with two hexadecimal characters, for example 0F.
Text can have empty lines and comment lines starting with /, ', # or : character.
Also after hexadecimal bytes there can be a comment character marking the rest of line as a comment.
Special keywords available:
STRING: | A normal string can be presented (without needing to type it in hex) |
INLINESTRING: | As STRING:, but Inline String token and termination null are automatically added |
LENGTH | Set this keyword to the place where the following bytes should be counted.
Next LENGTH keyword will place the counted number to the place where the first keyword was.
Nesting is not possible. |
See example below for more details.
|
Include |
Filename. Available since version 3.1.
Some parts of a message can be read from different file.
If an included file contains only text part, it should begin with one empty line.
|
Language Language_ext |
Available since version 3.1.16.
These settings define National Language Shift Tables to be used.
Text body must be written using UTF-8 character set.
Value can be number, or variable length string which first macthes (case insensitive).
Choices are:
- 0 = basic
- 1 = Turkish
- 2 = Spanish
- 3 = Portuguese
- 4 = Bengali and Assemese, or Bengali, or Assemese
- 5 = Gujarati
- 6 = Hindi
- 7 = Kannada
- 8 = Malayalam
- 9 = Oriya
- 10 = Punjabi
- 11 = Tamil
- 12 = Telugu
- 13 = Urdu
Usually it is not necessary to set Language_ext value.
When Language is set, Language_ext defaults to the same,
and if only Language_ext is defined, Language defaults to
basic character set. If nothing is set, default values are taken from the configuration.
|
Macro |
Definition. Available since version 3.1.
Syntax is: Macro: name=value.
Multiple macros can be defined.
All name's found in the message body are replaced with a value.
|
Message_reference |
Number. Available since version 3.1.16.
Sets TP-MR field in the PDU. Number can be 0...255.
|
Ping |
Boolean value. Available since version 3.1.16.
Selects the Short Message Type 0, which is also known as a silent SMS.
As this kind of SMS is not stored by the receiving device, report
is always requested, even if report was disabled in the configuration.
|
Priority |
Available since version 3.0. Possible value is:
High | Message is handled first when moving to spooler and when taking from spooler to sending |
|
Provider Queue |
Name of the provider, can be used to override the normal sorting algorithm configured by [providers] and [queues] in the config file. Both keywords do the same. |
Reject_duplicates |
Boolean value. Available since version 3.1.16.
Sets TP-RD bit in the PDU.
|
Replace |
Numeric code 1...7. Available since version 3.0.9.
If a receiving device and SIM supports "Replace Short Message Type n" -feature, a previously
received message with the same code is replaced with a new message. Only the messages sent from
the same originating address can be replaced. If there is nothing to replace, a message is stored
in the normal way.
|
Reply_path |
Boolean value. Available since version 3.1.16.
Sets TP-RP bit in the PDU.
|
Report |
Boolean value. Controls if a status report is requested for this message. Without this line, the setting from config file is used. |
Retries |
Number. Available since version 3.1.16.
Defines how many times smsd will retry if sending fails. This overrides send_retries setting.
|
SMSC |
Phone number of the SMSC. Since version 3.1 this setting is ignored if there is no smsc set in the config file. |
System_message |
Boolean value. Available since version 3.1.
With this setting message is sent as a system message.
This kind of message has fixed values 0x40 for Protocol Identifier and 0xF4 for Data Coding Scheme.
A message cannot have User Data Header.
Maximum length of a message is 140 bytes.
Since version 3.1.7, value for this setting can be 2 or ToSIM for communicating with SIM applications.
SMS is sent as SS (no show) and stored (sent) to SIM.
Currently this only works with binary messages.
|
Text_is_pdu |
Boolean value. Available since version 3.1.16.
If this feature is enabled by defining text_is_pdu_key in the configuration,
and To: number matches that key, message body is handled as ready made PDU.
|
To |
Receivers phone number in international format without the leading +.
When you like to send a message to a short number (for example to order a ringtone), then preceed it with an "s".
Since version 3.1 the number can be given using grouped format, like 358 12 345 6789.
|
To_TOA |
Available since version 3.1.5. Can be used to define receivers Type Of Address. This is also called "numbering plan". Possible values are
Unknown | No type is defined. Short numbers preceeded with "s" uses this value by default. |
International | Number is international. This is default for other than short numbers. |
National | Number is national. No country code is included. Some short numbers only work with this value. |
See Using Type Of Address selection for more details.
|
UDH |
Only binary messages: Boolean value, tells if the message data contains a user data header. Default is true. |
UDH-DATA |
User data header in hex-dump format. See udh.html and GSM 03.38.
Since version 3.1 also binary message can have UDH-DATA defined. |
Validity |
Available since version 3.0. Defines a message validity period.
Without this line, the setting from config file is used.
You can specify value as a number following one of the keywords: min, hour, day, week, month or year.
Validity period will be rounded down to the nearest possible value.
If you do not use any of those keywords, value will have the following meaning:
0 ... 143 | (value + 1) * 5 minutes (i.e. 5 minutes intervals up to 12 hours) |
144 ... 167 | 12 hours + ((value - 143) * 30 minutes) (i.e. 30 min intervals up to 24 hours) |
168 ... 196 | (value - 166) * 1 day (i.e. 1 day intervals up to 30 days) |
197 ... 255 | (value - 192) * 1 week (i.e. 1 week intervals up to 63 weeks) |
Incorrect values are ignored.
|
Voicecall |
Boolean value. Available since version 3.0.
With this feature the smsd will make a voice call to the receivers phone number given in To: header.
If the receiver answers to the call, some DTMF tones are played.
The message text must start with TONE: keyword. After this there can be number and space, which is number of times to repeat the tone sending.
Supported tones are #,*,0...9 and the tone list must be comma separated. For example:
TONE: 1,2,3,4,5,6,7,8,9,0
- this plays all digits, and it's repeated 3 times which is the default.
TONE: 5 #,#,#
- this plays three #'s, and it's repeated 5 times.
TONE:
- some default tones are played 3 times.
Since version 3.1 additional TIME: number definition can be used.
After a time has reached, hang up is done.
If a call is answered before a time is reached, normal sound playing is done.
NOTE that this time counting starts after a command is given to the modem and there will be
some delay before receiving device starts ringing.
You should test this with your own handset to find a reasonable time which works fine in the network you are using.
Example:
TONE: TIME: 15 2 #
Before using this feature to serious alarm purposes, you should test if this works with you modem/phone.
Also notice that automatic redialing should be turned off in the phone's settings.
After version 3.1.3 VTS command usage can be selected with voicecall_vts_list setting,
see the How to configure for more details.
After version 3.1.5 there is a new voicecall_ignore_modem_response setting for problematic devices,
see the How to configure for more details.
Also notice voicecall_hangup_ath setting if AT+CHUP does not hangup call on your device.
After version 3.1.7 there is a voicecall_cpas setting available.
If your device returns OK immediately after a dial command, with this setting AT+CPAS can be used to detect when a call is answered.
With this setting TIME: has no effect.
|
Note: In case of boolean values you can use true, yes, on or 1 for positive values. All other words are interpreted as negative.
After a message is sent, there will be automatically generated headers:
Message_id |
ID number of a sent message if a status report was requested.
Available since version 3.0.
|
Sent |
Time when the message was sent by the program.
Available since version 3.1.
|
Modem |
Name of the modem which was used to send this message.
Available since version 3.0.6.
|
IMSI |
International Mobile Subscriber Identity from the SIM,
if this request is supported.
Available since version 3.0.9.
|
IMEI |
IMEI of a modem which handled the message.
Available since version 3.1.16.
|
NOTICE |
Tells if some characters was not possible to convert to the target character set.
Available since version 3.1.16.
|
Sending_time |
Tells how long it took to send the whole message.
If the message has multiple parts and receive_before_send is set to yes,
time to receive messages is included in this value.
Available since version 3.1.20.
|
Binary data
The data begins after the empty line and goes until end of file. No conversion is applied to the data.
Example:
To: 491721234567
Alphabet: binary
UDH: true
gs2389gnsakj92"§Z/%$§"($)$(%ÄÖÜ?))((HJHG&(()/&§")(LJUFDZ)=W)==/685tgui
3ge^!"§$EGHWZFT&Z%8785ttghjjhdjkgfjsbfjwr793thruewgfh7328hgtwhg87324hf
hwer32873g&%§=)(/&%$%&/(/&%$§%&hdsgrwwq(/&%$fgzw543t43g5jwht934zt743g
|
Another example, Wap Push with Hex presentation, available since version 3.0:
To: 358401234567
Alphabet: binary
Hex: yes
// This is a sample Wap Push message:
06 : user Data Header Length (6 Octets)
05 : Identifer Element (16 Bit port addressing)
04 : Length of Parameter values (4 Octets)
# WAP Push connectionless session service (client side), Protocol: WSP/Datagram:
0B 84 : push dest port (2948)
# WAP connectionless session service Protocol: WSP/Datagram:
23 F0 : push originator port (9200)
01 : Push Transaction Id
06 : PDU Type Push, (WAP-230-WSP Table 34)
LENGTH // Headers Length will be placed to this position
AE : Push Header Content-Type: application/vnd.wap.sic 0x2E | 0x80
# (http://www.wapforum.org/wina/wsp-content-type.htm)
LENGTH // This stops the counting and places the number
02 : WBXML version 1.2
05 : SL 1.0 Public Identifier
04 : Charset=ISO-8859-1
00 : String table length
45 : si, with content
C6 : indication, with content and attributes
0D : Token for "href=http://www."
### There should not be extra spaces after keyword:
INLINESTRING:xyz
85 : Token for ".com/"
03 : Inline string follows
STRING:ppaid/123/abc.wml
00 : End of string
11 : si-id
INLINESTRING:1
01 : close of indication attribute list
INLINESTRING:Wap push demo from smstools3.
01 : End of indication element
01 : END of si element
# Specifications can be found from here:
#
http://www.openmobilealliance.org/tech/affiliates/wap/wapindex.html
|
Example of Wap Push with Macros and Include file:
In this example message body is the same for all messages, and it is saved as different file.
This file is included to the file which is sent by smstools. Macros are used.
Outgoing file contains only headers which are mandatory, and no message body at all.
File: /var/spool/sms/outgoing/wap_push1
To: 358401234567
Macro: @URL@=smstools3.kekekasvi.com/board.php
Macro: @TEXT@=Visit the SMS Server Tools 3 forum.
Include: /var/spool/sms/include/wap_push_service1
|
File: /var/spool/sms/include/wap_push_service1
Alphabet: binary
Hex: yes
06 05 04 0B 84 23 F0 01 06 01 AE 02 05
6A : Charset=UTF-8 (MIBEnum 106)
00 45 C6
0C : Token for "href=http://"
INLINESTRING:@URL@
11
INLINESTRING:1
01
INLINESTRING:@TEXT@
01 01
|
Received messages
The received SMS are stored in the same format as described above but they have some additional header lines. For example:
From: 491721234567
From_SMSC: 491722270333
Sent: 00-02-21 22:26:23
Received: 00-02-21 22:26:29
Subject: modem1
Alphabet: ISO
UDH: false
This is the Text that I have sent with my mobile phone to the computer.
|
Alphabet |
Tells the character set of the message text. |
Flash |
Boolean value. This header exists if a message was received as a flash (immediate display).
Note that usually phone devices do not save flash messages, they can be saved manually if necessary.
Available since version 3.1.
|
From |
Senders phone number. |
From_SMSC |
The SMS service centre, that sent you this message. |
From_TOA |
Type Of Address definition of senders phone number.
For example: "91 international, ISDN/telephone".
Available since version 3.0.9.
|
IMSI |
International Mobile Subscriber Identity from the SIM,
if this request is supported.
Available since version 3.0.9.
|
Length |
Length of text / data. With Unicode text number of Unicode characters.
If non-Unicode text message is stored using UTF-8, number of bytes may differ.
Available since version 3.1.
|
Received |
Time when the message was received by the program. |
Replace |
Replace Short Message Type 1..7 number, if defined.
Available since version 3.0.9.
|
Report |
Tells if a status report is going to be returned to the SME.
Available since version 3.0.9.
|
Sent |
Time when the message was sent. |
Subject |
The name of the modem that received this message. |
UDH |
Boolean value. Tells if the message contains a user data header. |
UDH-DATA |
This is the UDH in hex-dump format if the message contains an UDH. See udh.html and GSM 03.38. |
Since 3.0.9 there can be additional headers in case of some problems:
Error |
Tells if there was fatal errors and a message was not decoded.
Text part of message will tell more details and has no usual content. |
Warning |
Tells if there was minor proglems in the message data. |
The filenames of received messages look like modem1.xyzxyz. They begin with the name of the modem that received the message, followed by a dot, followed by six random characters.
Status Reports
You can request and receive status reports, if the SMSC and your modem support this feature. Example:
From: 491721234567
From_SMSC: 491722270333
Sent: 00-02-21 22:26:23
Received: 00-02-21 22:26:29
Subject: modem1
Alphabet: ISO
UDH: false
SMS STATUS REPORT
Message_id: 117
Discharge_timestamp: 00-02-21 22:27:01
Status: 0,Ok,short message received by the SME
|
Discharge_timestamp |
This is the time, when the message was successfully delivered or when it was discarded by the SMSC. |
Message_id |
This is the ID number of the previously sent message, where this status report belongs to.
The SMSC gives each sent message such a number. |
Status |
The status of the message. Please take a look into the source code src/pdu.c if you need a list of all possible status codes. |
Using Type Of Address selection
When SMSTools sends a message, it must tell to the Service Center what kind of number is used as a destination number.
This is called "Type Of Address". There are three possible values: "unknown", "international" and "national".
By default SMSTools assumes that:
- Any number without "s" is an international number.
- Numbers with "s" are "short" numbers and unknown Type Of Address is used.
However, all "short" numbers do not work with "unknown" type.
This is an operator depended issue and varies by country.
Because of this, version 3.1.5 and later has a new header To_TOA which can be used to manually define Type Of Address.
Type Of Address selection can be automated using two global settings in the configuration file: international_prefixes and national_prefixes.
Both settings are comma separated list of numbers.
If international_prefixes is defined, Type Of Address is international only if number matches to the list.
If it does not match, national Type Of Address is used.
If national_prefixes is defined, Type Of Address is national if number matches to the list.
And last, if there is To_TOA defined in the message file, this setting is used as it overrides everything else.
For example:
System is located in Finland and it's used to send messages mainly to Finland, Sweden and United Kingdom.
Both international and national number formats are used (for finnish numbers) and local short numbers should use national Type Of Address.
Global configuration setting international_prefixes = 358,46,44 is enough for normal operation.
With this setup, if SMS should be sent to Germany, it can be done with To_TOA: international header.
Another example:
Still in Finland. Any mobile phone number should work without additional headers.
Messages to the short numbers use "s" or will have To_TOA header if necessary.
Global configuration setting national_prefixes = 04, 05 or shortly national_prefixes = 0 is enough as finnish mobile numbers start with 04 or 05.
Messages to those numbers are sent using national Type Of Address, any other number will use international type.
|