code.delx.au - gnu-emacs-elpa/blob - packages/web-server/NOTES

   1                                                            -*- org -*-
   2 #+Title: Notes and Tasks
   3 #+HTML_HEAD: <style>pre{background:#232323; color:#E6E1DC;} table{margin:auto; width:50%;} @media(min-width:800px){div#content{max-width:800px; padding:2em; margin:auto;}}</style>
   4 #+Options: ^:{}
   5
   6 * Notes
   7 * Tasks [18/21]
   8 ** DONE Content and Transfer encodings
   9 - Content and Transfer encoding values
  10   http://www.iana.org/assignments/http-parameters/http-parameters.xhtml
  11 - http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6
  12 - http://en.wikipedia.org/wiki/Chunked_transfer_encoding
  13 - http://tools.ietf.org/html/rfc1945#section-7.2
  14 - http://tools.ietf.org/rfc/rfc1945.txt
  15
  16 Some issue as to whether compression is better done as a "Content
  17 Encoding" which actually changes the content, or as a "Transfer
  18 Encoding", which doesn't change the content, just the messages.
  19
  20 The latter seems preferable, but possibly less widely supported.  See
  21 http://stackoverflow.com/questions/11641923/transfer-encoding-gzip-vs-content-encoding-gzip.
  22
  23 - content-coding
  24   - compress :: Unix =compress= program (rfc2616)
  25   - deflate :: =zlib= (see http://www.iana.org/go/rfc1950) format with
  26        =defalte= compression (rfc2616)
  27   - exi :: W3c efficient XML (see http://www.w3.org/TR/exi/)
  28   - gzip :: GNU zip (rfc2616)
  29   - identity :: does nothing
  30   - pack200-zip :: specific to Java archives (see
  31        http://www.jcp.org/en/jsr/detail?id=200)
  32 - transfer-coding
  33   - chunked :: (rfc2616)
  34   - compress :: same as above
  35   - deflate :: same as above
  36   - gzip :: same as above
  37 - tail-header
  38   - Content-MD5 :: Base64 encoded binary MD5 sum of content
  39
  40 Maybe we can set the coding system of the process with
  41 =define-coding-system=, specifically using the =:pre-write-conversion=
  42 flag to e.g., gzip or chunkify the contents.
  43
  44 ** DONE web sockets
  45 - http://en.wikipedia.org/wiki/WebSocket
  46 - http://tools.ietf.org/html/rfc6455
  47
  48 ** more examples [4/4]
  49 *** DONE Org-mode agenda
  50 Already exists as part of org-ehtml.
  51 file:examples/011-org-agenda.el
  52
  53 *** DONE display the current buffer
  54 Idea stolen from elnode.
  55 file:examples/010-current-buffer.el
  56
  57 *** DONE browse the BBDB
  58 file:examples/012-search-bbdb.el
  59 *** DONE org-mode export server
  60 1. upload a file
  61 2. supply an export type
  62 3. return the exported version of the file
  63
  64 ** DONE handle large files
  65 When large files arrive quickly, the filter functions are called while
  66 they are still running on the previous chunk, this leads to nasty race
  67 conditions for the state of the request object.
  68
  69 Either introduce some check to wait on new input if input is currently
  70 being parsed, or wait until all input has arrived before doing any
  71 parsing.
  72
  73 Now using an =active= field on request objects to avoid race
  74 conditions when new header text is received while the parsing function
  75 is still active.
  76
  77 ** TODO robustness to bad requests [0/2]
  78 Low priority, just [[*running%20behind%20a%20proxy][run behind a proxy]].
  79
  80 *** TODO request timeout
  81 *** TODO maximum request size
  82 ** DONE authentication [2/2]
  83    - State "HOLD"       from "TODO"       [2014-02-10 Mon 19:06] \\
  84      digest may not be worth it, just run Basic over HTTPS
  85 *** DONE Basic
  86 http://en.wikipedia.org/wiki/Basic_access_authentication
  87
  88 *** CANCELED Digest
  89 http://en.wikipedia.org/wiki/Digest_access_authentication
  90
  91 If this is implemented, it would be good to implement some safeguards
  92 against common attacks.
  93
  94 #+begin_quote
  95 - Server nonce is allowed to contain timestamps. Therefore the server
  96   may inspect nonce attributes submitted by clients, to prevent replay
  97   attacks.
  98 - Server is also allowed to maintain a list of recently issued or used
  99   server nonce values to prevent reuse.
 100 #+end_quote
 101
 102 ** incremental handler calls
 103 not sure if the extra performance is worth the added complexity
 104
 105 Before the header is fully parsed, call any potential handlers.
 106 Include a field in the request object to indicate that the request
 107 isn't finished being received so handlers can return and wait to be
 108 called again.
 109
 110 Also, put a catch in the filter function and allow the =headers=
 111 function on the request object to throw to said catch aborting the
 112 handler and waiting for the rest of the input.
 113
 114 ** DONE Documentation [6/6]
 115 - [X] introduction
 116 - [X] handlers
 117 - [X] request headers
 118 - [X] usage examples
 119 - [X] list of functions
 120
 121 Notes to touch upon
 122 - [X] how to set content type
 123
 124 ** DONE Handle POST requests
 125 1. read standard for POST data
 126 2. parse multi-line headers with boundaries
 127
 128 For now keep this all incremental and in ws-filter.
 129
 130 ** DONE Makefile
 131 - byte-compile
 132 - package
 133 - test
 134 - benchmark
 135 ** DONE catch errors and return an error code
 136 include an easy error handler like the 404 handler
 137
 138 ** DONE better parsing of multipart form blocks
 139 parse more than just the content-type headers.
 140
 141 ** DONE non-multipart form data
 142 e.g., parameter strings
 143
 144 ** DONE some more convenience functionality [6/6]
 145 - [X] strip and parse URL query string
 146 - [X] parse urlencoded post data
 147 - [X] think about defaulting to (name . content) for form elements
 148 - [X] maybe don't require a non-nil return to cancel the connection,
 149       instead only keep open if =:keep-alive= is returned
 150 - [X] function to send a file (with mime handling)
 151 - [X] send a 404 with some default text
 152
 153 ** CANCELED Lazy header processing
 154    - State "CANCELED"   from "TODO"       [2013-12-25 Wed 12:21] \\
 155      premature optimization
 156 Use lazy sequence functions for header a-list to avoid parsing all
 157 headers.  For regexp matchers should stop when matched header is
 158 encountered (often the first one when :GET), For function matchers
 159 provide lazy version of assoc.
 160
 161 Also, there is the issue of how a lazy request for more parameters
 162 should act before all incoming text has been received.  Emacs does not
 163 provide a light-weight mechanism for a function to wait for incoming
 164 process text without something gross like the =(sit-for 0.1)= used in
 165 the test suite.
 166
 167 ** use gnutls for https
 168 low priority -- just [[*running%20behind%20an%20https%20proxy][run behind an https proxy]].
 169
 170 This will be a pain, and will require expanding [[info:emacs-gnutls]] to
 171 add support for starting server processes, currently only client
 172 processes are supported.
 173 ** screen cast?
 174 - http://en.wikipedia.org/wiki/XVidCap
 175 - https://aur.archlinux.org/packages/xvidcap/
 176
 177 * Tutorials
 178 The following tutorials walk through common usage scenarios including
 179 installing the Emacs web-server and running it behind a proxy.
 180 Install the Emacs web-server and run =(info "web-server")= to browse
 181 the full manual within Emacs, or view the HTML version at
 182 [[http://eschulte.github.io/emacs-web-server/][emacs-web-server]].
 183
 184 ** Installation and running a server
 185 Most easily installable through the GNU ELPA, run =M-x
 186 package-list-packages= select =web-server= and install.  Alternately,
 187 install from the git repository at
 188 https://github.com/eschulte/emacs-web-server and update your the load.
 189
 190 1. Ensure that you have Emacs version 24 or greater installed.
 191
 192    #+begin_src sh :results scalar
 193      emacs --version
 194    #+end_src
 195
 196    : GNU Emacs 24.3.1
 197    : Copyright (C) 2013 Free Software Foundation, Inc.
 198    : GNU Emacs comes with ABSOLUTELY NO WARRANTY.
 199    : You may redistribute copies of Emacs
 200    : under the terms of the GNU General Public License.
 201    : For more information about these matters, see the file named COPYING.
 202
 203 2. Download and unpack the zip archive of the Emacs web-server code
 204    from [[https://github.com/eschulte/emacs-web-server/archive/master.zip][emacs-web-server-master.zip]] or clone the source code
 205    repository with [[http://git-scm.com/][git]].
 206
 207    #+begin_src sh
 208      git clone https://github.com/eschulte/emacs-web-server.git
 209    #+end_src
 210
 211 3. Move into the root of the =emacs-web-server/= directory and
 212    optionally run =make= to compile the web-server code, and run =make
 213    check= to test your web-server install.
 214
 215    #+begin_src sh
 216      make
 217      make check
 218    #+end_src
 219
 220 4. From the root of the =emacs-web-server/= directory, start an
 221    instance of Emacs with web-server loaded.
 222
 223    #+begin_src sh
 224      emacs -Q -L . -l web-server
 225    #+end_src
 226
 227    Alternately, from an already running Emacs instance, add this
 228    directory to the load path and load the web server with the
 229    following.
 230
 231    #+begin_src emacs-lisp
 232      (add-to-list 'load-path "path/to/emacs-web-server")
 233      (require 'web-server)
 234    #+end_src
 235
 236 5. Evaluate the following code in =*scratch*= buffer of this Emacs
 237    instance.
 238
 239    #+begin_src emacs-lisp
 240      (ws-start
 241       (lambda (request)
 242         (with-slots (process headers) request
 243           (ws-response-header process 200 '("Content-type" . "text/plain"))
 244           (process-send-string process "hello world")))
 245       9000)
 246    #+end_src
 247
 248 6. Browse to http://localhost:9000 to see that the web-server is
 249    running.
 250
 251 7. Read the web-server [[http://eschulte.github.io/emacs-web-server/index.html#Top][manual]] and work through other [[http://eschulte.github.io/emacs-web-server/Usage-Examples.html#Usage-Examples][Usage Examples]].
 252
 253 ** Running behind a proxy
 254 Public-facing instance of the Emacs web-server should be run behind a
 255 more established web server such as [[http://httpd.apache.org/][Apache]] or [[http://wiki.nginx.org][Nginx]] to provide
 256 additional robustness and security.
 257
 258 The following example Apache configuration may be used to have a
 259 public facing Apache server listening on port 80 proxy requests to a
 260 local web-server instance running on port 8888 of the same machine.
 261
 262 #+begin_src conf
 263   <VirtualHost *:80>
 264     ServerName  yourserver.com
 265
 266     ProxyPass / http://localhost:8888/
 267   </VirtualHost>
 268 #+end_src
 269
 270 A similar Nginx configuration is available at
 271 http://wiki.nginx.org/LoadBalanceExample.
 272
 273 ** Running behind an https proxy
 274 The following example configurations will cause Apache or Nginx to act
 275 as an HTTPS proxy for an instance of the Emacs web server running on
 276 the same machine.  With this setup Apache speaks HTTPS to the outside
 277 world, and communicates with the Emacs web server using HTTP.  This
 278 allows use of HTTPS even though the Emacs web server does not
 279 implement HTTPS itself.  This setup is recommended for any setup, but
 280 should be considered *required* for sites using BASIC HTTP
 281 Authentication.
 282
 283 *** Apache
 284 This requires that Apache has =mod_proxy= and =mod_ssl= enabled, and
 285 that the certificate and key files required for SSL are present.  This
 286 these requirements satisfied, and assuming the Emacs web server is
 287 listening on port 8888 and is running on the same machine as the
 288 Apache web server an Apache virtual host configuration such as the
 289 following.
 290
 291 #+begin_src conf
 292   <VirtualHost *:443>
 293       ProxyPreserveHost On
 294       ServerName yourserver.com
 295
 296       SSLEngine On
 297       SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
 298       SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
 299
 300       ProxyPass / http://localhost:8888/
 301       ProxyPassReverse / http://localhost:8888/
 302   </VirtualHost>
 303 #+end_src
 304
 305 *** Nginx
 306 See the following for instructions configuring Nginx as an HTTPS
 307 proxy.
 308 - http://wiki.nginx.org/SSL-Offloader#sslproxy.conf
 309 - http://www.cyberciti.biz/faq/howto-linux-unix-setup-nginx-ssl-proxy/
 310
 311 ** COMMENT documentation for running in a chroot jail
 312 See https://wiki.archlinux.org/index.php/nginx#Installation_in_a_chroot.
 313
 314 * Bugs [1/1]
 315 ** DONE Sometimes servers don't stop cleanly
 316 - specifically servers with active client process
 317 - maybe also implement a =ws-stop-all= function