SES: Simple Emacs Spreadsheet
*****************************

     SES is a major mode for GNU Emacs to edit spreadsheet files, which
     contain a rectangular grid of cells.  The cells' values are specified
     by formulas that can refer to the values of other cells.

To report bugs, send email to jyavner@member.fsf.org.

This file documents SES: the Simple Emacs Spreadsheet.

Copyright © 2002--2013 Free Software Foundation, Inc.

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover texts being ``A GNU Manual,'' and with the Back-Cover Texts as in (a) below.  A copy of the license is included in the section entitled ``GNU Free Documentation License.''

     (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and modify this GNU manual.''

Sales Pitch
***********

  * Create and edit simple spreadsheets with a minimum of fuss.

  * Full undo/redo/autosave.

  * Immune to viruses in spreadsheet files.

  * Cell formulas are straight Emacs Lisp.

  * Printer functions for control of cell appearance.

  * Intuitive keystroke commands: C-o = insert row, M-o = insert column, etc.

  * ``Spillover'' of lengthy cell values into following blank cells.

  * Header line shows column letters or a selected row.

  * Completing-read for entering symbols as cell values.

  * Cut, copy, and paste can transfer formulas and printer functions.

  * Import and export of tab-separated values or tab-separated formulas.

  * Plaintext, easily-hacked file format.

The Basics
**********

To create a new spreadsheet, visit a nonexistent file whose name ends with ".ses".  For example, C-x C-f test.ses RET.

A cell identifier is a symbol with a column letter and a row number.  Cell B7 is the 2nd column of the 7th row.  For very wide spreadsheets, there are two column letters: cell AB7 is the 28th column of the 7th row. Super wide spreadsheets get AAA1, etc.

j
     Moves point to cell, specified by identifier (ses-jump).

Point is always at the left edge of a cell, or at the empty endline. When mark is inactive, the current cell is underlined.  When mark is active, the range is the highlighted rectangle of cells (SES always uses transient mark mode).  Drag the mouse from A1 to A3 to create the range A1-A2.  Many SES commands operate only on single cells, not ranges.

C-SPC
C-@
     Set mark at point (set-mark-command).

C-g
     Turn off the mark (keyboard-quit).

M-h
     Highlight current row (ses-mark-row).

S-M-h
     Highlight current column (ses-mark-column).

C-x h
     Highlight all cells (mark-whole-buffer).

Cell formulas
=============

To enter a number into the current cell, just start typing:

0..9
     Self-insert a digit (ses-read-cell).

-
     Self-insert a negative number (ses-read-cell).

.
     Self-insert a fractional number (ses-read-cell).

"
     Self-insert a quoted string.  The ending double-quote is inserted for you (ses-read-cell).

(
     Self-insert an expression.  The right-parenthesis is inserted for you (ses-read-cell).  To access another cell's value, just use its identifier in your expression.  Whenever the other cell is changed, this cell's formula will be reevaluated.  While typing in the expression, you can use M-TAB to complete symbol names.

' (apostrophe)
     Enter a symbol (ses-read-symbol).  SES remembers all symbols that have been used as formulas, so you can type just the beginning of a symbol and use SPC, TAB, and ? to complete it.

To enter something else (e.g., a vector), begin with a digit, then erase the digit and type whatever you want.

RET
     Edit the existing formula in the current cell (ses-edit-cell).

C-c C-c
     Force recalculation of the current cell or range (ses-recalculate-cell).

C-c C-l
     Recalculate the entire spreadsheet (ses-recalculate-all).

Resizing the spreadsheet
========================

Basic commands:

C-o
     (ses-insert-row)

M-o
     (ses-insert-column)

C-k
     (ses-delete-row)

M-k
     (ses-delete-column)

w
     (ses-set-column-width)

TAB
     Moves point to the next rightward cell, or inserts a new column if already at last cell on line, or inserts a new row if at endline (ses-forward-or-insert).

C-j
     Linefeed inserts below the current row and moves to column A (ses-append-row-jump-first-column).

Resizing the spreadsheet (unless you're just changing a column width) relocates all the cell-references in formulas so they still refer to the same cells.  If a formula mentioned B1 and you insert a new first row, the formula will now mention B2.

If you delete a cell that a formula refers to, the cell-symbol is deleted from the formula, so (+ A1 B1 C1) after deleting the third column becomes (+ A1 B1).  In case this is not what you wanted:

C-_
C-x u
     Undo previous action ((undo)).

Printer functions
=================

Printer functions convert binary cell values into the print forms that Emacs will display on the screen.

A printer can be a format string, like "$%.2f".  The result string is right-aligned within the print cell.  To get left-alignment, use parentheses: ("$%.2f").  A printer can also be a one-argument function (a symbol or a lambda), whose result is a string (right-aligned) or list of one string (left-aligned).  While typing in a lambda, you can use M-TAB to complete the names of symbols.

Each cell has a printer.  If nil, the column-printer for the cell's column is used.  If that is also nil, the default-printer for the spreadsheet is used.

p
     Enter a printer for current cell or range (ses-read-cell-printer).

M-p
     Enter a printer for the current column (ses-read-column-printer).

C-c C-p
     Enter the default printer for the spreadsheet (ses-read-default-printer).

The ses-read-XXX-printer commands have their own minibuffer history, which is preloaded with the set of all printers used in this spreadsheet, plus the standard printers.

The standard printers are suitable only for cells, not columns or default, because they format the value using the column-printer (or default-printer if nil) and then center the result:

ses-center
     Just centering.

ses-center-span
     Centering with spill-over to following blank cells.

ses-dashfill
     Centering using dashes (-) instead of spaces.

ses-dashfill-span
     Centering with dashes and spill-over.

ses-tildefill-span
     Centering with tildes (~) and spill-over.

Clearing cells
==============

These commands set both formula and printer to nil:

DEL
     Clear cell and move left (ses-clear-cell-backward).

C-d
     Clear cell and move right (ses-clear-cell-forward).

Copy, cut, and paste
====================

The copy functions work on rectangular regions of cells.  You can paste the copies into non-SES buffers to export the print text.

M-w
[copy]
[C-insert]
     Copy the highlighted cells to kill ring and primary clipboard (kill-ring-save).

[drag-mouse-1]
     Mark a region and copy it to kill ring and primary clipboard (mouse-set-region).

[M-drag-mouse-1]
     Mark a region and copy it to kill ring and secondary clipboard (mouse-set-secondary).

C-w
[cut]
[S-delete]
     The cut functions do not actually delete rows or columns---they copy and then clear (ses-kill-override).

C-y
[S-insert]
     Paste from kill ring (yank).  The paste functions behave differently depending on the format of the text being inserted:

       * When pasting cells that were cut from a SES buffer, the print text is ignored and only the attached formula and printer are inserted; cell references in the formula are relocated unless you use C-u.

       * The pasted text overwrites a rectangle of cells whose top left corner is the current cell.  If part of the rectangle is beyond the edges of the spreadsheet, you must confirm the increase in spreadsheet size.

       * Non-SES text is usually inserted as a replacement formula for the current cell.  If the formula would be a symbol, it's treated as a string unless you use C-u.  Pasted formulas with syntax errors are always treated as strings.

[paste]
     Paste from primary clipboard or kill ring (clipboard-yank).

[mouse-2]
     Set point and paste from primary clipboard (mouse-yank-at-click).

[M-mouse-2]
     Set point and paste from secondary clipboard (mouse-yank-secondary).

M-y
     Immediately after a paste, you can replace the text with a preceding element from the kill ring (ses-yank-pop).  Unlike the standard Emacs yank-pop, the SES version uses undo to delete the old yank.  This doesn't make any difference?

Customizing SES
===============

By default, a newly-created spreadsheet has 1 row and 1 column.  The column width is 7 and the default printer is "%.7g".  Each of these can be customized.  Look in group ``ses''.

After entering a cell value, point normally moves right to the next cell.  You can customize ses-after-entry-functions to move left or up or down.  For diagonal movement, select two functions from the list.

ses-mode-hook is a normal mode hook (list of functions to execute when starting SES mode for a buffer).

The variable safe-functions is a list of possibly-unsafe functions to be treated as safe when analyzing formulas and printers. See Virus protection.  Before customizing safe-functions, think about how much you trust the person who's suggesting this change.  The value t turns off all anti-virus protection.  A list-of-functions value might enable a ``gee whiz'' spreadsheet, but it also creates trapdoors in your anti-virus armor.  In order for virus protection to work, you must always press n when presented with a virus warning, unless you understand what the questionable code is trying to do.  Do not listen to those who tell you to customize enable-local-eval---this variable is for people who don't wear safety belts!

Advanced Features
*****************

C-c M-C-h
     (ses-set-header-row). The header line at the top of the SES window normally shows the column letter for each column.  You can set it to show a copy of some row, such as a row of column titles, so that row will always be visible.  Default is to set the current row as the header; use C-u to prompt for header row.  Set the header to row 0 to show column letters again.

[header-line mouse-3]
     Pops up a menu to set the current row as the header, or revert to column letters.

M-x ses-rename-cell
     Rename a cell from a standard A1-like name to any string.

M-x ses-repair-cell-reference-all
     When you interrupt a cell formula update by clicking C-g, then the cell reference link may be broken, which will jeopardize automatic cell update when any other cell on which it depends is changed. To repair that use function ses-repair-cell-reference-all

The print area
==============

A SES file consists of a print area and a data area.  Normally the buffer is narrowed to show only the print area.  The print area is read-only except for special SES commands; it contains cell values formatted by printer functions.  The data area records the formula and printer functions, etc.

C-x n w
     Show print and data areas (widen).

C-c C-n
     Show only print area (ses-renarrow-buffer).

S-C-l
M-C-l
     Recreate print area by reevaluating printer functions for all cells (ses-reprint-all).

Ranges in formulas
==================

A formula like

     (+ A1 A2 A3)

is the sum of three specific cells.  If you insert a new second row, the formula becomes

     (+ A1 A3 A4)

and the new row is not included in the sum.

The macro (ses-range FROM TO) evaluates to a list of the values in a rectangle of cells.  If your formula is

     (apply '+ (ses-range A1 A3))

and you insert a new second row, it becomes

     (apply '+ (ses-range A1 A4))

and the new row is included in the sum.

While entering or editing a formula in the minibuffer, you can select a range in the spreadsheet (using mouse or keyboard), then paste a representation of that range into your formula.  Suppose you select A1-C1:

[S-mouse-3]
     Inserts "A1 B1 C1" (ses-insert-range-click)

C-c C-r
     Keyboard version (ses-insert-range).

[C-S-mouse-3]
     Inserts "(ses-range A1 C1)" (ses-insert-ses-range-click).

C-c C-s
     Keyboard version (ses-insert-ses-range).

If you delete the FROM or TO cell for a range, the nearest still-existing cell is used instead.  If you delete the entire range, the formula relocator will delete the ses-range from the formula.

If you insert a new row just beyond the end of a one-column range, or a new column just beyond a one-row range, the new cell is included in the range.  New cells inserted just before a range are not included.

Flags can be added to ses-range immediately after the TO cell.

!
     Empty cells in range can be removed by adding the ! flag. An empty cell is a cell the value of which is one of symbols nil or *skip*. For instance (ses-range A1 A4 !) will do the same as (list A1 A3) when cells A2 and A4 are empty.

_
     Empty cell values are replaced by the argument following flag _, or 0 when flag _ is last in argument list. For instance (ses-range A1 A4 _ "empty") will do the same as (list A1 "empty" A3 "empty") when cells A2 and A4 are empty. Similarly, (ses-range A1 A4 _ ) will do the same as (list A1 0 A3 0).

>v
     When order matters, list cells by reading cells row-wise from top left to bottom right. This flag is provided for completeness only as it is the default reading order.

<v
     List cells by reading cells row-wise from top right to bottom left.

v>
     List cells by reading cells column-wise from top left to bottom right.

v<
     List cells by reading cells column-wise from top right to bottom left.

v
     A short hand for v>.

^
     A short hand for ^>.

>
     A short hand for >v.

<
     A short hand for >^.

*
     Instead of listing cells, it makes a Calc vector or matrix of it (see GNU Emacs Calc Manual). If the range contains only one row or one column a vector is made, otherwise a matrix is made.

*2
     Same as * except that a matrix is always made even when there is only one row or column in the range.

*1
     Same as * except that a vector is always made even when there is only one row or column in the range, that is to say the corresponding matrix is flattened.

Sorting by column
=================

C-c M-C-s
     Sort the cells of a range using one of the columns (ses-sort-column).  The rows (or partial rows if the range doesn't include all columns) are rearranged so the chosen column will be in order.

[header-line mouse-2]
     The easiest way to sort is to click mouse-2 on the chosen column's header row (ses-sort-column-click).

The sort comparison uses string<, which works well for right-justified numbers and left-justified strings.

With prefix arg, sort is in descending order.

Rows are moved one at a time, with relocation of formulas.  This works well if formulas refer to other cells in their row, not so well for formulas that refer to other rows in the range or to cells outside the range.

Standard formula functions
==========================

Oftentimes you want a calculation to exclude the blank cells.  Here are some useful functions to call from your formulas:

(ses-delete-blanks &rest ARGS)
     Returns a list from which all blank cells (value is either nil or '*skip*) have been deleted.

(ses+ &rest ARGS)
     Sum of non-blank arguments.

(ses-average LIST)
     Average of non-blank elements in LIST.  Here the list is passed as a single argument, since you'll probably use it with ses-range.

More on cell printing
=====================

Special cell values:

  * nil prints the same as "", but allows previous cell to spill over.

  * '*skip* replaces nil when the previous cell actually does spill over; nothing is printed for it.

  * '*error* indicates that the formula signaled an error instead of producing a value: the print cell is filled with hash marks (#).

If the result from the printer function is too wide for the cell and the following cell is nil, the result will spill over into the following cell.  Very wide results can spill over several cells.  If the result is too wide for the available space (up to the end of the row or the next non-nil cell), the result is truncated if the cell's value is a string, or replaced with hash marks otherwise.

SES could get confused by printer results that contain newlines or tabs, so these are replaced with question marks.

t
     Confine a cell to its own column (ses-truncate-cell).  This allows you to move point to a rightward cell that would otherwise be covered by a spill-over.  If you don't change the rightward cell, the confined cell will spill over again the next time it is reprinted.

c
     When applied to a single cell, this command displays in the echo area any formula error or printer error that occurred during recalculation/reprinting (ses-recalculate-cell).  You can use this to undo the effect of t.

When a printer function signals an error, the fallback printer "%s" is substituted.  This is useful when your column printer is numeric-only and you use a string as a cell value.  Note that the standard default printer is ``%.7g'' which is numeric-only, so cells that are empty of contain strings will use the fallback printer. c on such cells will display ``Format specifier doesn't match argument type''.

Import and export
=================

x t
     Export a range of cells as tab-separated values (ses-export-tsv).

x T
     Export a range of cells as tab-separated formulas (ses-export-tsf).

The exported text goes to the kill ring; you can paste it into another buffer.  Columns are separated by tabs, rows by newlines.

To import text, use any of the yank commands where the text to paste contains tabs and/or newlines.  Imported formulas are not relocated.

Virus protection
================

Whenever a formula or printer is read from a file or is pasted into the spreadsheet, it receives a ``needs safety check'' marking.  Later, when the formula or printer is evaluated for the first time, it is checked for safety using the unsafep predicate; if found to be ``possibly unsafe'', the questionable formula or printer is displayed and you must press Y to approve it or N to use a substitute.  The substitute always signals an error.

Formulas or printers that you type in are checked immediately for safety.  If found to be possibly unsafe and you press N to disapprove, the action is canceled and the old formula or printer will remain.

Besides viruses (which try to copy themselves to other files), unsafep can also detect all other kinds of Trojan horses, such as spreadsheets that delete files, send email, flood Web sites, alter your Emacs settings, etc.

Generally, spreadsheet formulas and printers are simple things that don't need to do any fancy computing, so all potentially-dangerous parts of the Emacs Lisp environment can be excluded without cramping your style as a formula-writer.  See the documentation in unsafep.el for more info on how Lisp forms are classified as safe or unsafe.

Spreadsheets with details and summary
=====================================

A common organization for spreadsheets is to have a bunch of ``detail'' rows, each perhaps describing a transaction, and then a set of ``summary'' rows that each show reduced data for some subset of the details.  SES supports this organization via the ses-select function.

(ses-select FROMRANGE TEST TORANGE)
     Returns a subset of TORANGE.  For each member in FROMRANGE that is equal to TEST, the corresponding member of TORANGE is included in the result.

Example of use:

     (ses-average (ses-select (ses-range A1 A5) 'Smith (ses-range B1 B5)))

This computes the average of the B column values for those rows whose A column value is the symbol 'Smith.

Arguably one could specify only FROMRANGE plus TO-ROW-OFFSET and TO-COLUMN-OFFSET.  The TORANGE is stated explicitly to ensure that the formula will be recalculated if any cell in either range is changed.

File etc/ses-example.el in the Emacs distribution is an example of a details-and-summary spreadsheet.

For Gurus
*********

Deferred updates
================

To save time by avoiding redundant computations, cells that need recalculation due to changes in other cells are added to a set.  At the end of the command, each cell in the set is recalculated once. This can create a new set of cells that need recalculation.  The process is repeated until either the set is empty or it stops changing (due to circular references among the cells).  In extreme cases, you might see progress messages of the form ``Recalculating... (NNN cells left)''.  If you interrupt the calculation using C-g, the spreadsheet will be left in an inconsistent state, so use C-_ or C-c C-l to fix it.

To save even more time by avoiding redundant writes, cells that have changes are added to a set instead of being written immediately to the data area.  Each cell in the set is written once, at the end of the command.  If you change vast quantities of cells, you might see a progress message of the form ``Writing... (NNN cells left)''. These deferred cell-writes cannot be interrupted by C-g, so you'll just have to wait.

SES uses run-with-idle-timer to move the cell underline when Emacs will be scrolling the buffer after the end of a command, and also to narrow and underline after C-x C-v.  This is visible as a momentary glitch after C-x C-v and certain scrolling commands.  You can type ahead without worrying about the glitch.

Nonrelocatable references
=========================

C-y relocates all cell-references in a pasted formula, while C-u C-y relocates none of the cell-references.  What about mixed cases?

You can use

     (symbol-value 'B3)

to make an absolute reference.  The formula relocator skips over quoted things, so this will not be relocated when pasted or when rows/columns are inserted/deleted.  However, B3 will not be recorded as a dependency of this cell, so this cell will not be updated automatically when B3 is changed.

The variables row and col are dynamically bound while a cell formula is being evaluated.  You can use

     (ses-cell-value row 0)

to get the value from the leftmost column in the current row.  This kind of dependency is also not recorded.

The data area
=============

Begins with an 014 character, followed by sets of cell-definition macros for each row, followed by column-widths, column-printers, default-printer, and header-row.  Then there's the global parameters (file-format ID, numrows, numcols) and the local variables (specifying SES mode for the buffer, etc.)

When a SES file is loaded, first the numrows and numcols values are loaded, then the entire data area is evaled, and finally the local variables are processed.

You can edit the data area, but don't insert or delete any newlines except in the local-variables part, since SES locates things by counting newlines.  Use C-x C-e at the end of a line to install your edits into the spreadsheet data structures (this does not update the print area, use, e.g., C-c C-l for that).

The data area is maintained as an image of spreadsheet data structures that area stored in buffer-local variables.  If the data area gets messed up, you can try reconstructing the data area from the data structures:

C-c M-C-l
     (ses-reconstruct-all).

Buffer-local variables in spreadsheets
======================================

You can add additional local variables to the list at the bottom of the data area, such as hidden constants you want to refer to in your formulas.

You can override the variable ses--symbolic-formulas to be a list of symbols (as parenthesized strings) to show as completions for the ' command.  This initial completions list is used instead of the actual set of symbols-as-formulas in the spreadsheet.

For an example of this, see file etc/ses-example.ses.

If (for some reason) you want your formulas or printers to save data into variables, you must declare these variables as buffer-locals in order to avoid a virus warning.

You can define functions by making them values for the fake local variable eval.  Such functions can then be used in your formulas and printers, but usually each eval is presented to the user during file loading as a potential virus.  This can get annoying.

You can define functions in your .emacs file.  Other people can still read the print area of your spreadsheet, but they won't be able to recalculate or reprint anything that depends on your functions.  To avoid virus warnings, each function used in a formula needs

     (put 'your-function-name 'safe-function t)

Uses of defadvice in SES
========================

undo-more
     Defines a new undo element format (FUN . ARGS), which means ``undo by applying FUN to ARGS''.  For spreadsheet buffers, it allows undos in the data area even though that's outside the narrowing.

copy-region-as-kill
     When copying from the print area of a spreadsheet, treat the region as a rectangle and attach each cell's formula and printer as 'ses properties.

yank
     When yanking into the print area of a spreadsheet, first try to yank as cells (if the yank text has 'ses properties), then as tab-separated formulas, then (if all else fails) as a single formula for the current cell.

Acknowledgments
***************

Coding by:

     Jonathan Yavner jyavner@member.fsf.org

     Stefan Monnier monnier@gnu.org

     Shigeru Fukaya shigeru.fukaya@gmail.com

Texinfo manual by:

     Jonathan Yavner jyavner@member.fsf.org

     Brad Collins <brad@chenla.org>

Ideas from:

     Christoph Conrad christoph.conrad@gmx.de

     CyberBob cyberbob@redneck.gacracker.org

     Syver Enstad syver-en@online.no

     Ami Fischman fischman@zion.bpnetworks.com

     Thomas Gehrlein Thomas.Gehrlein@t-online.de

     Chris F.A. Johnson c.f.a.johnson@rogers.com

     Yusong Li lyusong@hotmail.com

     Juri Linkov juri@jurta.org

     Harald Maier maierh@myself.com

     Alan Nash anash@san.rr.com

     François Pinard pinard@iro.umontreal.ca

     Pedro Pinto ppinto@cs.cmu.edu

     Stefan Reichör xsteve@riic.at

     Oliver Scholz epameinondas@gmx.de

     Richard M. Stallman rms@gnu.org

     Luc Teirlinck teirllm@dms.auburn.edu

     J. Otto Tennant jotto@pobox.com

     Jean-Philippe Theberge jphil@acs.pagesjaunes.fr

GNU Free Documentation License
******************************

     [file doclicense.texi not included]