Generating headings for project files

I use the free software AutoHotkey (AHK) for automation of many things. One of those things is generating headings for my project file. I developed a simple AHK-script that asks me for the project name, the topic and then inserts the heading with additional information like my name, my E-mail address and a keyword for subversion.

The script is quite simple and I have adopted it for headings for several project files (tex, R and gams). The only difference being the characters used to comment out a line.

Here is the script

The line with ::tplt:: shows the characters that when typed start the template script (in this case for a tex-file). The lines with “Inputbox” produce a box where I can enter information

message

In this case, I enter the project name. What follows is the input box for the subject and then some additional information. Note, that the command “send” needs a variable (between %-values), therefore, everything  without variables should be on the same line.

Here the result for a new gams project file:

gams

You can easily adapt this script for your own use.

Automatic operator formatting mode in Emacs

Emacs has a nice new mode called electric-operator developed by David Spepherd. It helps when you write code by formatting all operators in a predefined way. For example, in R it adds spaces around the operator signs, when you write 1+1, this is automatically converted into 1 + 1.

You can define your own way of formatting for other modes. For gams-mode you add the following to your .emacs file

First you set gams-mode as a mode on which electric operator can work.Then you add some rules. After that you set a hook so electric-operator is automatically started when you use Gams (or ESS).

A special formatting is necessary for the comments in GAMS as they start with the same symbol as the multiplication. With the help of David, I could add a function gams-mode-* that changes the “*” used as a comment in “* “ and otherwise in “ * “.  It checks if the “*” symbol is at the beginning of the line and in that case will not add a space in front of it.

It probably needs some more tweaking, and I will keep you posted on improvements.

Using file templates in Emacs

A good practice in modeling is to place information related to the project your are working on, the subject, the data and your personal information like E-mail address at the top of your file. In Emacs you can define skeleton functions that will prompt for the information and put it in the file. Below is an example for a file with as comment symbol “**: (the $Id: $ is for my version control system, that automatically will add the information on the last commit in this line).

image

Below you see how to make a function using skeleton. As I don’t want to repeat myself, the function asks as an input the comment symbol(s) that should be used. I now can use the function for different programming languages (R uses “#” , C++ uses two forward slashes, GAMS “*”, etc.). You can add the time and date by using the command “current-time-string”. All other inputs that have skeleton-read in front of it, are also prompted for. If you need the template for your whole project team, you could just make the author and contact field also input-dependent.

image 

More information can be found at: SkeletonMode. (note that this is part of Emacs since version 22).

PS. The function puts an extra comment symbol at the end (not shown). This is a bug in my function. Any ideas how to get rid of this are welcome.

Automating writing LaTeX papers even further

In an earlier post I wrote on how to use R to produce all kind of figures and tables for a LaTeX paper. This time, I will show, how to automate this process even more.

My work flow consists of a batch file, that runs my model, sends the results to R, produces my tables and figures and sends these to my paper: If I change my model, I just have to run this batch file, and the results in my paper are automatically adjusted. One smaller problem is that I often write in my text on a specific value from a table (for example, the minimum time used). Every time I change my model, I have to check these entries by hand (of course, I could use knitr, but that is on my list for the future and at the moment, I want to keep my R-files separate from my LaTeX files).

Here is a neat trick, to automate this process. Suppose I have calculated the reduction in time in my model and exported the results to R. In my paper I have the following  sentence:

“As you can see in Table 1,in the scenario “Cruising” the reduction in driving time is 12.4%.”

In my R-script, I have the R-Code that produces this value or takes the value from my model results directly. Now I use some simple R-commands to produce a LaTeX file that can be included in my paper:

In my LaTeX-paper, I now just change the sentence to:

This produces the same sentence as before. However, if I change my model, the minimal reduction is changed and automatically adjusted.

Using org-mode in outline-mode

I have written about using org-mode in gams-mode in an earlier post. Gams-mode lets you structure your gams file using headings and subheadings. These headings can be (partially) collapsed. The structure of your file could look like this:

image

You can now cycle through the structure using Shift-Tab. Tab opens a section:

image

This is a great help in complex files.

Gams-mode has another great outline feature for the listing file. Instead of putting a normal comment with a “*” in front, one writes a display statement that starts with a “@” (see the code above). We now see a nice heading in our listing file.

image

However, what is really fantastic, is the display of the actual loop state in the outline mode (in the listing file you can easily search for the loop element, but in the outline mode this is not possible). Now you can see the loop status in the above code after the “LOO” . Hitting “space” the listing will be shown in another window.

Migrating to R 3.0.0: TikzDevice and gdxrrw

I am an avid user of R. Most of the time, I use R for preparing nice plots and tables for my LaTeX papers. For my plots, I use TikzDevice and for my tables I use xtable. Gams, my modeling software, developed a nice interface for using data from Gams in R and the other way around (see http://support.gams.com/doku.php?id=gdxrrw:interfacing_gams_and_r). When I upgraded from R 2.15 to R 3.0, I had some problems on my windows 8 machine, especially with TikZDevice. I finally managed to get it installed. The necessary trick is described on “The Stata Things” (!!!, http://enoriver.net/index.php/2013/04/12/keeping-knitr-happy-after-upgrading-to-r-3-0-0/ )
Here is what you have to do, to get TikzDevice up and running:
  • Install R 3.0
  • Update your packages, using the command: update.packages(checkBuilt=TRUE)
  • Install Rtools (http://cran.r-project.org/bin/windows/Rtools/) and adjust your path (sytem variable PATH) by adding the bin-directory of Rtools (something like C:\Program Files\Rtools\bin)
  • Download tikzdevice from http://r-forge.r-project.org/R/?group_id=440
  • Run the following commands in the R Console
  • remove.packages(“tikzDevice”)
  • install.packages(“Path to where the file on your computer is/tikzDevice_0.6.3.tar.gz”, repos = NULL, type=”source”)
For Gams users: download the gdxrrw 0.3.0 (not 0.2.0) from the homepage (http://support.gams.com/doku.php?id=gdxrrw:interfacing_gams_and_r ). Extract the files in your R library and add the following line to your Rprofile.site (in the directory etc of your R-installation), where you have to set the gams directory properly.
.First <- function() {
     library(gdxrrw)
     a <- R.Version()$arch
     if (a == “x86_64”)  igdx(“C:/program files/gams_64/23.9.4”)
     else                igdx(“C:/program files/gams_32/23.9.4”)
   }
PS. I like the fortunes package. It produces nice quotations when you start up R. Just install the package and add the following lines to your Rprofile.site
   if (interactive())
   fortunes::fortune()
Here an example
I wish to perform brain surgery this afternoon at 4pm and don’t know where to start. My background is the history of great statistician sports legends but I am willing to learn. I know there are courses and numerous books on brain surgery but I don’t have the time for those. Please
direct me to the appropriate HowTos, and be on standby for solving any problem I may encounter while in the operating room. Some of you might ask for specifics of the case, but that would require my following the posting guide and spending even more time than I am already taking to
write this note.
  — I. Ben Fooled (aka Frank Harrell)
     R-help (April 1, 2005)

GAMS and Excel: writing and checking results with automatic closing and opening

I often use the Gams utility gdxxrw to write the results of my model simulations to Excel. In the testing phase, if you don’t know some nice tricks, this means a lot of closing and opening Excel: If the result file is open in excel, you get an error, when you try to write results to the Excel file, so you have to close Excel first and rerun your simulation. After writing the results to Excel you have to reopen the file again. Lots of keystrokes…
However, gdxxrw lets you reduce the number of strokes, by using the gdxxrw utility xlstalk.
 
For the problem above, you could write the following:
 
       execute ‘xlstalk -S results.xlsx’
       execute ‘gdxxrw.exe results.xlsx i=results.gdx ….’
       execute ‘xlstalk -O results.xlsx’
 
The first line saves the file results.xlsx and closes it (if open) and closes Excel. The second line is the command line for writing the results to the file. The last line opens the file in Excel.
Using this lines, you can run your model over and over again without manually closing and opening Excel.

Using Windows Desktop Search to search for text in files with another extension than "txt"

I use Windows Search quite often on my notebook to look for a file, E-Mail or a program. In Windows you just hit the Windows Key and start typing, which is very convenient.
If you want to start a program you just start typing the program name and hit enter Excel, hit the key and type “exc” and enter, and off you go.
The normal search (files or text within files) is also very powerful. Here is a site which explains the syntax: http://msdn.microsoft.com/en-us/library/aa965711(v=vs.85).aspx and here is a good article http://www.howtogeek.com/73065/learn-the-advanced-search-operators-in-windows-7/ on how to use it.
 
One problem with Windows Search is, that it doesn’t do a text search in my Gams files. I was looking for a file with the word “CPLEX” in it. I knew, I wrote that file last year, but it did not show up in my search. The solution is telling Windows Desktop Search that it should also index the file contents of Gams files. You can do this by going to your Control Panel, click on Indexing Options and then on Advanced Options. Look for the “gms” extension and change the indexing form “Index Properties Only” to “Index Properties and File Contents” (see below). I also removed a lot of file types, I never want to search for.
 
 
 
 
Here is an example of the windows desktop search in Windows 8 (I searched for the term “urban” in Gams files):
 
 
 
 

Outlining in Gams-Mode: Org-Mode for Gams

The new version of Gams-mode has a new feature that I like very much. It allows you to use an outline-mode in your gams files. It is called org-mode, which is a major mode in Emacs.
 
You can outline your Gams file using sections, subsections, etc. You define sections using the following notation:
 
       *@
 
or for subsections
 
       *@@, *@@@, etc.
 
Let us have a look at the file org-minor-mode.gms. This file can be found in your gams-mode directory. This file contains all the help for gams-mode. All the help topics have a section title that starts with *@.
Here is an overview of the contents of this file:
 
 
This is great! Now, instead of scrolling or searching, you can just cycle through the different display possibilities using Shift-Tab (org-global-cycle), or if you want to see or hide the content of a certain section, hit Tab (org-cycle).
 
Here, I used Tab to show the subsections of the section “Process handling”
 
And if I hit Tab again on the first subsection, it shows the contents :
 
 

Gams code snippet for checking consistency of a matrix

In an earlier post, I showed how to include code snippets in your program. One of the most important principles of programming is the DRY principle: Don’ Repeat Yourself. Code snippets can be used to do stuff you have to code over and over again.
One such a task for me is checking if the row and column totals in a matrix are equal (for example in the input output table you use, you want to check if supply is equal to demand).
 
Here is the code snippet:
 
 
Now you can batinclude (meaning that you can pass additional information to this included file with arguments %1, %2 etc.).
 
Here a simple example of how to use this code snippet:
 
 
Some remarks:
1. The code snippet in this file is saved in the file sn_checkcons.gms in the same directory as the other files.
You might want to have a directory that is accessible for all your projects as I do.
 
I just add the following code at the beginning of the project file:
 
 
The gams snippets are saved in the directory D:\Snippets\Gams and I can include the snippets with
 
 
where the three dots are the arguments.
 
2. The arguments are the set used and the name of the matrix you want to check. In the code snippet everywhere gams finds %1 this is replaced with the set name (in the example “i”) and %2 with the matrix name (in the example “matrix”).