summaryrefslogtreecommitdiff
path: root/data/docs/11
blob: 202d90243b19a9c4d18aae08b1a8eb71dba31391 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
:TITLE:Public Database API

<div class="warning">
 <h2>IN DEVELOPMENT</h2>
 <p>This API is currently in development, pretty much everything is subject to change without notice.</p>
</div>

:INC:index

:SUB:Introduction
<p>
 This document describes the public API of VNDB and is intended to be read by
 programmers. The API allows programs and websites to access (parts of) the VNDB
 database without actually visiting a page on the website.
 <br /><br />
</p>

<b>Design goals</b>
<ul>
 <li>
  Simple in implementation of both client and server. "Simple" here means that
  it shouldn't take much code to write a secure and full implementation and that
  client applications don't require huge dependency trees just to use this API.
 </li>
 <li>Powerful: Not as powerful as raw SQL, but not as rigid as commonly used REST or RPC protocols.</li>
 <li>Fast: minimal bandwidth overhead</li>
 <li>
  High-level: nobody is interested in the internal database structure of VNDB
  (ok, maybe you are, but you wouldn't want to write an application using it: it
  changes quite often)
 </li>
 <li>Stateful</li>
</ul>
<br />

<b>Design overview</b>
<ul>
 <li>TCP-based, all communication between the client and the server is done
  using one TCP connection. This connection stays alive until it is explicitely
  closed by either the client or the server.</li>
 <li>Request/response, client sends a request and server replies with a response.</li>
 <li>Session-based: clients are required to login before issuing commands to
  the server. A session is created by issuing the 'login' command, this session
  stays valid for the lifetime of the TCP connection.</li>
 <li><b>Everything</b> sent between the client and the server is encoded in UTF-8.</li>
</ul>
<br />

<b>Limits</b>
<p>The following limits are enforced by the server, in order to limit the
server resources and prevent abuse of this service.</p>
<ul>
 <li>5 connections per IP. All connections that are opened after reaching this limit will be immediately closed.</li>
 <li>3 connections per user. The login command will reply with a 'sesslimit' error when reaching this limit.</li>
 <li>Each command currently returns at most 10 results. <i>TODO: make configurable?</i></li>
 <li><i>more to come...</i></li>
</ul>

<br />
<b>Test version:</b>
<dl>
 <dt>Host (beta)</dt><dd>beta.vndb.org</dd>
 <dt>Port</dt><dd>19534 ('VN')</dd>
</dl>


:SUB:Request/response syntax
<p>
 The VNDB API uses the JSON format for data in various places, this document assumes
 you are familiar with it. See <a href="http://json.org/">JSON.org</a> for a quick
 overview and <a href="http://www.ietf.org/rfc/rfc4627.txt?number=4627">RFC 4627</a>
 for the glory details.
 <br /><br />
 The words <i>object</i>, <i>array</i>, <i>value</i>, <i>string</i>,
 <i>number</i> and <i>int</i> refer to the JSON data types. In addition the following
 definitions are used in this document:
</p>
<dl>
 <dt><i>request</i> or <i>command</i></dt><dd>
  Message sent from the client to the server.
 </dd><dt><i>response</i></dt><dd>
  Message sent from the server to the client.
 </dd><dt><i>whitespace</i></dt><dd>
  Any sequence of the following characters: space, tab, line feed and carriage
  return.  (hexadecimal: 20, 09, 10, 0D, respectively). This is in line with the
  definition of whitespace in the JSON specification.
 </dd><dt><i>date</i></dt><dd>
  A <i>string</i> signifying a date (in particular: release date). The
  following formats are used: "yyyy" (when day and month are unknown), "yyyy-mm"
  (when day is unknown) "yyyy-mm-dd", and "tba" (To Be Announced). If the year is
  not known and the date is not "tba", the special value <b>null</b> is used.
 </dd>
</dl>
<br />

<b>Message format</b>
<p>
 A message is formatted as a command or response name, followed by any number of
 arguments, followed by the End Of Transmission character (04 in hexadecimal).
 Arguments are separated by one or more whitespace characters, and any sequence
 of whitespace characters is allowed before and after the message.<br />
 The command or response name is an unescaped string containing only lowercase
 alphabetical ASCII characters, and indicates what kind of command or response
 this message contains.<br />
 An argument can either be an unescaped string (not containing whitespace), any
 JSON value, or a filter string. The following two examples demonstrate a
 'login' command, with an object as argument. Both messages are equivalent, as
 the whitespace is ignored. '0x04' is used to indicate the End Of Transmission
 character.
</p>
<pre>
 login {"protocol":1,"username":"ayo"}<b class="standout">0x04</b>
</pre><pre>
 login {
  "protocol" : 1,
  "username" : "ayo"
 }
 <b class="standout">0x04</b>
</pre>
The 0x04 byte will be ommitted in the other examples in this document. It is
however still required.<br />

<br />
<b>Filter string syntax</b>
<p>
 Some commands accept a filter string as argument. This argument is formatted
 similar to boolean expressions in most programming languages. A filter consists
 of one or more <i>expressions</i>, separated by the boolean operators "and" and
 "or" (lowercase). Each filter expression can be surrounded by parentheses to
 indicate precedence, the filter argument itself must be surrounded by parentheses.
 <br />
 An <i>expression</i> consists of a <i>field name</i>, followed by an
 <i>operator</i> and a <i>value</i>. The field name must consist entirely of
 lowercase alphanumeric characters and can also contain an underscore. The
 operator must be one of the following characters: =, !=, &lt;, &lt;=, &gt;,
 &gt;= or ~.  The <i>value</i> can be any valid JSON value. Whitespace
 characters are allowed, but not required, between all expressions, field names,
 operators and values.<br />
 The following two filters are equivalent:
</p>
<pre>
 (title~"osananajimi"or(id=2))
</pre><pre>
 (
   id = 2
   or
   title ~ "osananajimi"
 )
</pre>
<p>More complex things are also possible:</p>
<pre>
 ((platforms = ["win", "ps2"] or languages = "ja") and released > "2009-01-10")
</pre>
<p>See the individual commands for more details.</p>


:SUB:The 'login' command
<pre>
 login {"protocol":1,"client":"test","clientver":0.1,"username":"ayo","password":"hi-mi-tsu!"}
</pre>
<p>
 Every client is required to login before issuing other commands. The login
 command accepts a JSON object as argument. This object must have the following members:
</p>
<dl>
 <dt>protocol</dt><dd>An integer that indicates which protocol version the client implements. Must be 1.</dd>
 <dt>client</dt><dd>
  A string identifying the client application. Between the 3 and 50 characters,
  must contain only alphanumeric ASCII characters, space, underscore and hyphens.
  When writing a client, think of a funny (unique) name and hardcode it into
  your application.
 </dd><dt>clientver</dt><dd>A positive number indicating the software version of the client.</dd>
 <dt>username</dt><dd>String containing the username of the person using the client.</dd>
 <dt>password</dt><dd>String, password of that user in plain text.</dd>
</dl>
<p>
 The server replies with either 'ok' (no arguments), or 'error' (see below).
</p>


:SUB:The 'get' command
<p>
 This command is used to fetch data from the database. It accepts 3 arguments:
 the type of data to fetch (e.g. visual novels or producers), what part of that
 data to fetch (e.g. only the VN titles, or the descriptions and relations as
 well), and lastly a filter expression.
</p>
<pre>
 get <b class="standout">type flags filters</b>
</pre>
<p>
 Type and flags are unescaped strings. The only type currently accepted is 'vn'.
 Flags is a comma-separated list of flags indicating what info to fetch. The
 filters, available flags and their meaning are documented separately for each
 type. The following example will fetch basic information and information about
 the related anime of the visual novel with id = 17:
</p>
<pre>
 get vn basic,anime (id = 17)
</pre>
<p>
 The server will reply with a 'results' message, this message is followed by a
 JSON object describing the results. This object has two members: 'num', which
 is an integer indicating the number of results returned, and 'items', which
 contains the results as an array of objects. For example, the server could
 reply to the previous command with the following message:
</p>
<pre>
 results {"num":1, "items":[{
   "id": 17, "title": "Ever17 -the out of infinity-", "original": null,
   "released": "2002-08-29", "languages": ["en","ja","ru","zh"],
   "platforms": ["drc","ps2","psp","win"],"anime": []
 }]}
</pre>
<p>
 Note that the actual result from the server can (and likely will) be formatted
 differently and that the order of the members may not be the same. What each
 member means and what possible values they can have differs per type and is
 documented below.
</p><br />

<b>Returned object members for the 'vn' type:</b>
<table style="margin: 5px 2%; width: 95%">
 <thead><tr>
  <td style="width: 80px">Member</td>
  <td style="width: 50px">Flag</td>
  <td style="width: 90px">Type</td>
  <td style="width: 40px">null</td>
  <td>Description</td>
 </tr></thead>
 <tr class="odd">
  <td>id</td>
  <td>-</td>
  <td>integer</td>
  <td>no</td>
  <td>Visual novel ID</td>
 </tr>
 <tr>
  <td>title</td>
  <td>basic</td>
  <td>string</td>
  <td>no</td>
  <td>Main title</td>
 </tr>
 <tr class="odd">
  <td>original</td>
  <td>basic</td>
  <td>string</td>
  <td>yes</td>
  <td>Original/official title.</td>
 </tr>
 <tr>
  <td>released</td>
  <td>basic</td>
  <td>date (string)</td>
  <td>yes</td>
  <td>Date of the first release.</td>
 </tr>
 <tr class="odd">
  <td>languages</td>
  <td>basic</td>
  <td>array of strings</td>
  <td>no</td>
  <td>Can be an empty array when nothing has been released yet.</td>
 </tr>
 <tr>
  <td>platforms</td>
  <td>basic</td>
  <td>array of strings</td>
  <td>no</td>
  <td>Can be an empty array when unknown or nothing has been released yet.</td>
 </tr>
 <tr class="odd">
  <td>aliases</td>
  <td>details</td>
  <td>string</td>
  <td>yes</td>
  <td>Comma-separated list of aliases.</td>
 </tr>
 <tr>
  <td>length</td>
  <td>details</td>
  <td>integer</td>
  <td>yes</td>
  <td>Length of the game, 1-5</td>
 </tr>
 <tr class="odd">
  <td>description</td>
  <td>details</td>
  <td>string</td>
  <td>yes</td>
  <td>Description of the VN. Can include formatting codes as described in <a href="/d9.3">d9.3</a>.</td>
 </tr>
 <tr>
  <td>links</td>
  <td>details</td>
  <td>object</td>
  <td>no</td>
  <td>
   Contains the following members:<br />
   "wikipedia", string, name of the related article on the English Wikipedia.<br />
   "encubed", string, the URL-encoded tag used on <a href="http://novelnews.net/">encubed</a>.<br />
   "renai", string, the name part of the url on <a href="http://renai.us/">renai.us</a>.<br />
   All members can be <b>null</b> when no links are available or known to us.
  </td>
 </tr>
 <tr class="odd">
  <td>anime</td>
  <td>anime</td>
  <td>array of objects</td>
  <td>no</td>
  <td>
   (Possibly empty) list of anime related to the VN, each object has the following members:<br />
   "id", integer, <a href="http://anidb.net/">AniDB</a> ID<br />
   "ann_id", integer, <a href="http://animenewsnetwork.com/">AnimeNewsNetwork</a> ID<br />
   "nfo_id", string, <a href="http://animenfo.com/">AnimeNfo</a> ID<br />
   "title_romaji", string<br />
   "title_kanji", string<br />
   "year", integer, year in which the anime was aired<br />
   "type", string<br />
   All members except the "id" can be <b>null</b>. Note that this data is courtesy of AniDB,
   and may not reflect the latest state of their information due to caching.
  </td>
 </tr>
 <tr>
  <td>relations</td>
  <td>relations</td>
  <td>array of objects</td>
  <td>no</td>
  <td>
   (Possibly empty) list of related visual novels, each object has the following members:<br />
   "id", integer<br />
   "relation", string, relation to the VN<br />
   "title", string, (romaji) title<br />
   "original", string, original/official title, can be <b>null</b>.
  </td>
 </tr>
</table>

<b>Accepted filters for the 'vn' type:</b>
<table style="margin: 5px 2%; width: 95%">
 <thead><tr>
  <td style="width: 80px">Field</td>
  <td style="width: 90px">Value</td>
  <td style="width: 90px">Operators</td>
  <td>&nbsp;</td>
 </tr></thead>
 <tr class="odd">
  <td>id</td>
  <td>integer<br />array of integers</td>
  <td>= != &gt; &gt;= &lt; &lt;=<br />= !=</td>
  <td>
   When you need to fetch info about multiple VNs, it is recommended to do so
   in one command using an array of integers as value. e.g. (id = [7,11,17]).
  </td>
 </tr>
 <tr>
  <td>title</td>
  <td>string</td>
  <td>= != ~</td>
  <td>&nbsp;</td>
 </tr>
 <tr class="odd">
  <td>original</td>
  <td>null<br />string</td>
  <td>= !=<br />= != ~</td>
  <td>&nbsp;</td>
 </tr>
 <tr>
  <td>released</td>
  <td>null<br />date (string)</td>
  <td>= !=<br />= != &gt; &gt;= &lt; &lt;=</td>
  <td>
   Note that matching on partial dates (released = "2009") doesn't do what
   you want, use ranges instead, e.g. (released > "2008" and released <= "2009").
  </td>
 <tr class="odd">
  <td>platforms</td>
  <td>null<br />string<br />array of strings</td>
  <td><br />= !=</td>
  <td>&nbsp;</td>
 </tr>
 <tr>
  <td>languages</td>
  <td>null<br />string<br />array of strings</td>
  <td><br />= !=</td>
  <td>&nbsp;</td>
 </tr>
 <tr class="odd">
  <td>search</td>
  <td>string</td>
  <td>~</td>
  <td>
   This is not an actual field, but performs a search on the titles of the visual
   novel and its releases. Note that the algorithm of this search may change and
   that it can use a different algorithm than the search function on the website.
  </td>
</table>



:SUB:The 'error' response
<p>
 Every command to the server can receive an 'error' response, this response has one
 argument: a JSON object containing at least a member named "id", which identifies
 the error, and a "msg", which contains a human readable message explaining what
 went wrong. Other members are also possible, depending on the value of "id".
 Example error message:
</p>
<pre>
 error {"id":"parse", "msg":"Invalid command or argument"}
</pre>
<p>
 Note that the value of "msg" is not directly linked to the error identifier:
 the message explains what went wrong in more detail, there are several
 different messages for the same id. The following error identifiers are currently
 defined:
</p>
<dl>
 <dt>parse</dt><dd>Syntax error or unknown command.</dd>
 <dt>missing</dt><dd>A JSON object argument is missing a required member. The name of which is given in the additional "field" member.</dd>
 <dt>badarg</dt><dd>A JSON value is of the wrong type or in the wrong format. The name of the incorrect field is given in a "field" member.</dd>
 <dt>needlogin</dt><dd>Need to be logged in to issue this command.</dd>
 <dt>auth</dt><dd>(login) Incorrect username/password combination.</dd>
 <dt>loggedin</dt><dd>(login) Already logged in. Only one successful login command can be issues on one connection.</dd>
 <dt>sesslimit</dt><dd>(login) Too many open sessions for the current user.</dd>
 <dt>gettype</dt><dd>(get) Unknown type argument to the 'get' command.</dd>
 <dt>getinfo</dt><dd>(get) Unknown info flag to the 'get' command. The name of the unrecognised flag is given in an additional "flag" member.</dd>
 <dt>filter</dt><dd>(get) Unknown filter field or invalid combination of field/operator/argument type. Includes three additional members: "field", "op" and "value" of the incorrect expression.</dd>
</dl>