Using code snippets in Gams: batinclude

One of the important rules of good programming is the DRY principle: Don’t Repeat Yourself. After repeating myself too often, I decided to start working with code snippets and macros for often used code. Here is an example of code I use very often. Suppose you want to aggregate your data from your input-output table to more aggregated version, for example, from 87 sectors to only 5 sectors, so you can test your model more easily). In GAMS you do this with a mapping, where you map the 87 sectors to the 5 sectors. This mapping can then be used for aggregating your IOT easily.
What often happens is that you forget to map one element, or you map one element twice. Here is a simple example:
 
 
We have two sets: the first set “dis” is the disaggregated  and the second set “agg” is the aggregated set. You want to map the first set to the second:
 
 
The elements “a” and “b” for set “dis” are mapped to “1” from set “agg” and “c” to “2”. Assume that in you make a mistake and forget to map “c” and you map “b” twice:
 
 
In “real world” work this can happen very easily. Tom Rutherford wrote some nice code to check for errors like these. As I always use this error checking for mappings and I don’t want to repeat myself, I wrote the following code snippet saved in the file sn_checkmapping.gms. Try to understand what is going on here (I won’t try to explain it) or just use if for your work.
 
 
Checking the mapping from my example is now easy (I don’t have to adjust any code):
 
$batinclude sn_checkmapping dis agg mapping
 
The two sets and the mapping are used as arguments (%1,%2 and %3) in the code snippet. The output in your listing file will look like this:
 
 
We forgot to map c and mapped b twice.
 
You can save code snippets and batinclude files like that in a common directory and use it in all your projects.
 
 

Auto-Completion in Emacs, LaTeX and Gams

The fun with Emacs is that you can discover new (or old) stuff that makes your life much easier on an almost weekly base. I was reading the manual on Emacs Speaks Statistics and discovered that there is a mode called auto-completion. I googled it, installed it and after some experimenting, I got it running. You can find it here. It comes with autocompletion for some programming languages (Python, Ruby, C+, etc.) but not for LaTeX, so I googled further and discovered an extension called auto-complete-latex (you can download it here.
Here are some screenshots: You just start typing and it suggests a keyword or if you wait a little bit longer it gives you a dropdown llist. You can jump to the next suggestions using the tab-key.
It doesn’t matter if you use capitals or not:
and it also keeps track of words in the actual document. Here an example of my paper on parking. As you see the word “Parking appears, when I typ “par”.
For Gams there is no dictionary, so I wrote one myself and added it to the dictionaries. I only did not manage to start auto-complete for Gams automatically. But this can be done with M-x auto-complete-mode.
Update:
;; ac-modes – major modes ac can run on
(add-to-list ‘ac-modes ‘gams-mode)

;; toggle auto-complete in all buffers

(global-auto-complete-mode t)

(with thanks to a reader)
These extensions make Emacs now to a modern text editor.

Using Gams, R and LaTeX I

Wow, I just discovered how easy it is to produce nice LaTeX tables from your Gams results. Usually I capture my model results in some parameters that are exported (using gdx or gdxxrw) to excel. Then I make my tables in Excel and use an excel add-in to export the table to LaTeX (I have written about this add-in in this blog). This works fine, but is tedious because as soon as my results are in excel, I can’t do things automatically and have to click my way to get my LaTeX tables.
Today, I was working on the parking model and wanted to automate my LaTeX tables. I already wrote on how to produce nice plots using R Statistics, but that might have been too geeky for a lot of you (“I don’t do network modelling and Excel or Gnuplot produces plots that will do…”). However, what I show you today is worth trying to start working with R if you are doing your modeling with Gams.
My workflow now looks like this:
  1. Run my model and produce some parameter with results and export the parameters to a gdx file
  2. Read the parameters from the gdx file with R using the R package (gdxrrw).
  3. Recast the data using “cast” and “mold” from the package reshape (see for an excellent introduction (http://www.jstatsoft.org/v21/i12/paper)
  4. Use the R package “xtable” (see http://cran.r-project.org/web/packages/xtable/xtable.pdf) to produce for each table a separate tex file.
  5. Write my paper and include the tex files with tables.
Let us have a look at the code for every step.
1. Run my model and produce some parameter with results  and export the parameters to a gdx file
Most of this is straight forward. Note that I save my gdx files in my R directory. In this directory I have my R-scripts.
2. Read the parameters from the gdx file with R using the R package (gdxrrw).
Gams Corporation is working on a R-package that allows you to transfer data from R to Gams and the other way around. The package is called gdxrrw (the “rrw” stands for R read and write)  is still work-in-progress, but you can already use it (just download it from (http://support.gams.com/doku.php?id=gdxrrw:interfacing_gams_and_r).
When you have it installed, you can use it as follows:
The gams parameter is now a R data frame with the following structure:
3. Recast the data using “cast” and “mold” from the package reshape
I want to produce two different tables: one for the model statistisc and one for the road results. I saved all this results in one parameter, so now I have to take a subset of the results for every table.
I defined a subset “modelstatistics” and use them to subset my dataframe using %in%. This gives us the following dataframe
I want to have a table with the parameter as columns and the scenarios as rows. For this I first rename the columns and then I use the cast command:
If I combine this with the xtable command, I already have my LaTeX table:
4. Use the R package “xtable” to produce for each table a separate tex file.
Now we are almost ready: I set some options for saving the LaTeX table (table title, label and filename) and save the file using the print command on the table:
5. Write my paper and include the tex files with tables.
I jump in my paper tex file and add the following line for including the table:
and produce my LaTeX file:
In your xtable-code in the R-script you can a lot of additonal options to format your table.
The great thing is that I can do it all now without having to adjust my tables every time I run a different or addtional scenario. I just add in gams the following line to run the R-script:
And for the Emacs fans: I didn’t have to leave Emacs at all: I run my Gams model, my R code and my LaTeX from within Emacs (and for those with the good eyes: I can even clock this task…)
In a next post, I will write more on the use of cast: you can use this R command for lots of interesting stuff (building tables with the sum, mean, min, max, etc.).

Using the identifier-list in Gams-mode

Gams-mode has lots of features and I am still amazed how powerful it is. I just discovered the use of the identifier-list (C-c C-a). I have a rather complex model with lots of include files and easily forget all the parameter names I use. Was it scale_mrd or mrd_scale? What was the name of the set HH or HHG? Here is where the identifier list comes in handy. Just hit C-c C-a and all the parameters, sets, variables appear as a new buffer.
 
 
 
 
You can change the way this information is listed. For example hit in this buffer “j” and all the information is grouped. You can hit the “+” or “-” sign to show or hide a certain group. Now you can work in your gams file and scroll or search through the identifier list. It is also helpful if you have forgotten to write the text describing an identifier: just scroll through the list and use “space” to jump to the identifier in you gams file and add the description. If you hit “enter” you jump to the identifier and the identifier-list buffer is closed.
 

Template Mode for Emacs

If you write a lot of R-, Stata-, Gams and/or LaTeX-files templates can make your life more relaxed. Emacs doesn’t have a template system (well there is auto-insert, but I did not find enough information on how to use if for templates). I used to have some templates in a directory and usually I would start with them (and forget that I should save it under another name…). The disadvantage is that you have to jump around in the file to fill out the information you want to have at the top of the file (like project name, topic, file name etc.). Today I discovered templatemode (see http://emacs-template.sourceforge.net/details.html). It works like a breeze and makes my writing life much easier.
 
I have templates for the above mentioned file types. The Gams template looks like this:
 
Now if I start a new file, TemplateMode asks me if I want to use the Gams-template, it inserts all the information between >>> and <<< or asks for information to insert. For example DIR, DATE will grab the directory and the date. At the end of the template file you see pairs of words. These words define the interactive part. Templatemode asks me for the project name with Project:, the job with Job: and the Gams title with Title:.
 
The line with the Version Control Information grabs information from my verison control system as soon as I commit the file (and is not part of templatemode).
 
This is how a new Gams file would look like:
 
 
Installation is easy and you can find lots of examples in the directory.
 
 

Update of Gams-Tex

Edson Valle sent me an update for Gams-Tex (see my blog from 28. Januari) where he showed how to add new keywords to the Gams-Tex file (here done for the MPSGE keywords):
 
Just add the following lines to the gams theme file (theme_gams.tex) after the moreKeywords={
$PROD, $DEMAND, $AUXILIARY, $COMMODITIES, $SECTORS, $CONSUMERS, $MODEL, $REPORT,
It’ll become automatically case insensitive due to the options selected.

New update of Gams-Mode (3.6)

There is a update for Gams-mode. It contains some bug fixes and a nice new feature: If you hit C-u C-c C-. you can search for an identifier. It finds the parameter and has several options like jump to the next occurrence, jump to the file, split the screen and also show the file where gams-mode found the parameter. All these possibilities are described at the bottom-line and are self-explaining (“decl” shows the declaration of the identifier).
 
 
 

Gams code in LaTeX documents

Edson Valle recently wrote a style for the package listings from LaTeX. It allows you to format Gams code nicely in your LaTeX documents.
Here is a small example taken from his post to the Gams mailing list:
Scalar X /1/;
Scalar Y;
Y = 2*X;
Display “This is the famous Y”, Y;
In your LaTeX document this would look like:
The only thing you have to do is write a file gams.tex with the style and include this file in your LaTeX document before \begin{document}.
Information on the package Listings can be found here: listings
The gams.tex file should look like this:
\usepackage{listings}
%author Edson Cordeiro do Valle edsoncv at gmail dot com

\lstdefinelanguage{GAMS}{
morekeywords={
ABORT , ACRONYM , ACRONYMS , ALIAS , ALL , AND , ASSIGN , BINARY , CARD , DISPLAY , EPS , EQ , EQUATION , EQUATIONS , GE , GT , INF , INTEGER , LE , LOOP , LT , MAXIMIZING , MINIMIZING , MODEL , MODELS , NA , NE , NEGATIVE , NOT , OPTION , OPTIONS , OR , ORD , PARAMETER , PARAMETERS , POSITIVE , PROD , SCALAR , SCALARS , SET , SETS , SMAX , SMIN , SOS1 , SOS2 , SUM , SYSTEM , TABLE , USING , VARIABLE , VARIABLES , XOR , YES , REPEAT , UNTIL , WHILE , IF , THEN , ELSE , SEMICONT , SEMIINT , FILE , FILES , PUT , PUTPAGE , PUTTL , PUTCLOSE , FREE , NO , SOLVE , FOR , ELSEIF , ABS , ARCTAN , CEIL , COS , ERROR , EXP , FLOOR , LOG , LOG10 , MAP , MAPVAL , MAX , MIN , MOD , NORMAL , POWER , ROUND , SIGN , SIN , SQR , SQRT , TRUNC , UNIFORM , LO , UP , FX , SCALE , PRIOR , PC , PS , PW , TM , BM , CASE , DATE , IFILE , OFILE , PAGE , RDATE , RFILE , RTIME , SFILE , TIME , TITLE , TS , TL , TE , TF , LJ , NJ , SJ , TJ , LW , NW , SW , TW , ND , NR , NZ , CC , HDCC , TLCC , LL , HDLL , TLLL , LP , WS , /,PROD: },
sensitive = false,
morecomment=[f]*,%
morecomment=[s]{$ontext}{$offtext},
morecomment=[s][\color{green}]{/}{/},
morestring=[b]”,
morestring=[b]’
}
\lstset{
basicstyle=\fontfamily{pcr}\fontseries{m}\selectfont\footnotesize,
commentstyle=\color{gray}\itshape,
keywordstyle=\color{blue}\bfseries,
stringstyle=\color[rgb]{0.5,0,0.5}\itshape,
showstringspaces=false,
numbers=left,
numberstyle=\color[rgb]{0,0.5,0.5}\fontfamily{pcr}\fontseries{m}\selectfont\tiny,
numberblanklines=false,
showlines=false,
belowskip=\bigskipamount{},
breaklines=true,
%stepnumber=2,
tabsize=6,
%extendedchars=true,
%float=h,
frame=tb
}

Using the Gams Manuals in Gams-Mode

You can call the help and solver files from within Gams.mode hitting C-c Enter. Be sure that the directory for the help files is properly set (choose Customize Gams mode for Emacs from the Gams menu) and search for the following entry in the buffer:
 
If you hit enter, the gams manual is shown in your PDF viewer, but you can also search for another file. For example: type path and hit tab for completion: Gams-mode will show you two completions for ‘path’: PATH-Solver and PATH-LP-Solver.
You should also set the path of the PDF file viewer (the variable gams-docs-view-programs):
 

Do not forget to save the settings, which you can do at the beginning of the settings buffer.
You can set them for the current or/and for all future settings:
 

Indenting with Gams-mode

The new version of Gams-mode has a nice feature that helps you in formatting your model.
You can automatically indent GAMS programs according to GAMS syntax.
 
M-C-\        =       Indent region.  Specify region and type M-C-\.
TAB          =       Indent the current line.
 
On the left you see the model before using the indent feature of Gams-mode. I selected the region and typed M-C-\ (`indent-region’).
You can see on the right side what happened. The set and parameter declarations are nicely formatted and the loop is clearly visible because all assignments in the loop are indented.
This example was taken from the file gams-sample.gms which resides in the gams-mode directory (and contains lots of useful tricks with Gams-mode).
 
 
 
Another nice feature is indenting a region (examples taken from the sample-text.gms file from Gams-mode.
Just mark the region you wan to indent and hit c-c c-y. Choose “t” if it is a table and “o” for other text.
Here an example (once again taken from the sample file):
 
or a table