TSM Blog

30/07/10
Version 4.32, released today, has few new features apparent to the user, but represents a major code reorganization. The graphics code, formerly part of the graphical user interface, has now been included in the kernel. This means that it can be accessed by users' command-line Ox programs that import TSM as a module. The programming manual has a new section describing how to implement the new graphics functions. The Gnudraw code module is now compiled as part of trmknl4.oxo, which accordingly is the only TSM component needing to be imported to access TSM functions.
All these changes should be entirely transparent to the user of the GUI. However, a new version of Gnudraw allows some enhancements to the data-plotting options, basically extending multiple-series plotting to the correlogram, partial correlogram, spectrum and QQ-representations of series. Also new is user control over graphics text point size and font, although the latter setting is effective only for exported graphics files. Interactive graphics have the Arial font by default, but this can, as before, be edited in the Gnuplot window by clicking the top-left frame icon.
Among minor interface revisions, note that the file type selector for graphics files has been moved to the "Options / Output and Retrieval" dialog, to accommodate the new items in Options / Graphics. Also, from the same dialog it is now possible to list the gradient of the criterion function in the output. This feature may have occasional diagnostic uses. There are also many other small fixes and enhancements too small to document. Hopefully this release is now reasonably clean, but bug reports will be gratefully received, as always.          

30/04/10
The Values dialogs have had a redesign. The "Refresh", "Clear" and "Next..." buttons are now located on their own panel below the scroll-pane. This saves the need to scroll the dialogs in the case when there are a large number of parameters in the model. 

28/03/10
This release clarifies the format for reporting cointegration (ADF and PP) tests. It also updates the LM test for parameter fixes, which are now computed with analytic derivatives whenever available. This is an extremely useful feature that allows the test of virtually any model specification against virtually any more general alternative. The procedure is very simple. Given any fitted model, extend its specification to the alternative of interest. Then, open the relevant Values dialog, and check the Fixed checkbox against the newly specified parameters, with their values set to zero. Next, select Actions /  Compute Test Statistics / LM Tests of Parameters Restrictions, and hit OK, after checking out the cautionary message - note that the test will only be valid if it is computed at the restricted estimates, so these must not be changed. A way to ensure this is to check the "LM Tests of Parameter Restrictions" checkbox in Options / Tests and Diagnostics. Then re-estimate with the restrictions imposed, and the test is reported automatically. If restrictions are set up through the Model / Parameter Constraints dialog (check the "Restricted Estimation" radio button) the test will be conducted on these restrictions instead. This option is handy if some parameters need to be fixed under the alternative. 

10/03/10
Today's release features listing and plotting of partial correlograms. It is also possible to set the number of correlogram points (regular or partial) to be plotted, by setting the scrollbar in the Summary Statistics dialog. It also sorts out some problems with creating EPS graphics files, and there has been some redesign of dialogs. A problem with asymmetric (threshold) Garch has been sorted. These models need numerical derivatives to be selected for estimation and this was not happening automatically, as it should. Analytic volatility forecasts for asymmetric Garch models have also been corrected.     

12/12/09
Version 4.31, released today, only has a couple of new features. The Elliott-Rothenberg-Stock (1996) efficient tests of I(1), and also the Robinson-Lobato (1998) test of I(0), can now be computed through the Summary Statistics dialog.  There is an option to include either the squares or the absolute values of regressors in a conditional variance equation. (Previously, absolute values was the only choice.) Otherwise, this release is mainly a consolidation of many minor fixes and improvements. The algorithm for DCC Garch has been improved. A new release of  GnuDraw is incorporated, which fixes some issues related to graphing of confidence bands, and the 'filled' options for conditional variances, probabilities and similar plots. The 'fill' feature now works generally, previously the plots needed to have dates for the option to be implemented. As detailed in the last blog post, discrete data options now include the ordered probit and ordered logit models.    

29/09/09
This is the latest of a series of releases during September reflecting our polishing of the Condor capability, and the fixing of various issues that came to light. Analytic derivatives for the skew-Student ML are now in place.
The sample can now be independently selected for each of the various program functions - estimation/simulation, plotting, editing and transformations, summary statistics and so forth. To save confusion, the current selection is now displayed at the top-left of each of the relevant dialogs, except for the estimation/simulation sample which appears as usual on the status bar.
A new capability in this release, though not major enough to justify incrementing the version number, is ordered probit and logit estimation. The models are parameterized in such a way as to make it straightforward to identify the model in the presence of empty categories.

12/08/09
A further bug-fix release posted today.

24/07/09
Whoops!  The initial post of v4.30 was a dud, unfortunately. Some experimental code stubs not removed, so everything was taking a year to optimize. Apologies, and please upgrade to v4.30.24-07-09 without delay. 

21/07/09
While it may not appear so, Version 4.30 is quite a major upgrade. A long-standing ambition to implement analytic derivatives for the search algorithm has now been largely implemented. This should result in faster and more reliable optimization, especially for the larger and more complicated models. More accurate calculation of the Hessian matrix of the criterion function is a useful spinoff. The implementation is not yet completed and some models, most notably the Markov-switching class, EGARCH models, and the skew-Student and GED maximum likelihood estimators, still use the old method. We intend that the capability will be extended to all cases in future releases. Since our code has not yet been thoroughly tested in all situations, a fail-safe procedure compares numerical and analytic derivatives, at the default parameter values, at the start of each optimization. The numerical option is used in the event of a discrepancy. These changes should be seamless to the user, although a message in the output indicates that the numerical option has been used, for whatever reason.  However, in case of an unanticipated problem a checkbox in the Options / Optimization and Run dialog allows the numerical option to be reinstated as the default.

Another innovation allows jobs to be run on the Condor HTC (high-throughput computing) system, on networks where this facility is installed. Monte Carlo jobs can be split into any number of parallel instances to be run simultaneously on otherwise idle workstations on a network. Aggregating these results gives the effect of a running a single job on an arbitrarily fast processor. On a large university network, experiments could run hundreds of times faster than on a conventional desktop. This option is still experimental, and the current implementation is for Windows-based networks only. Feedback from users with access to a Condor installation will be most welcome. 

23,27/06/09
This release implements (finally) the White heteroscedasticity test, in LM and conditional moment variants. The opportunity has been taken to redesign the Options / Tests and Diagnostics dialog, which was already overcrowded and incapable of accommodating any more options. Now, there is a separate dialog for Diagnostic Tests, allowing ample space for new tests to be added in the future. The new dialog can be accessed with a button in the simplified Tests and Diagnostics dialog. However, it is compact enough to keep open alongside the Linear Regression and Dynamic Equation dialogs, and these  are equipped with new buttons to access it directly. The result is, hopefully, a more direct and intuitive model specification interface.    

20,30/03/09
These updates have fixed a problem with the latest code, and also restore a feature that got lost in the update to OxJapi2 - this is the ability to plot a series by choosing it from any list and pressing the "Plot" button. 

12/02/09
This update of v4.29 has an enhanced Model menu that includes quick links to the basic Model Manager functions, saving a model and loading from a list of available models. By default, up to 20 model names are displayed at a time, with the most recently loaded appearing first. This is different from the list in Model Manager, which can include up to 100 models listed by order of creation, although these can optionally be rearranged.  

14/01/09
Version 4.29 (Windows version) supports file drag-and-drop. To load a data set (or several data sets) simply drag them from Windows Explorer over the TSM window, and release. Settings (.tsm) files, model (.tsd) and tabulation files can be loaded in the same manner. 

There is also a change in the operation of error message display, following an invalid action by the user. The message box now has a "Continue" button that needs to be clicked to close it. Previously, the message box was displayed for two seconds and then closed automatically. The latter behaviour still applies to message boxes displaying simple information.     

17/18/21-12-08
There have been several recent updates, to fix various small problems that came to light with the new code. The current posting is hopefully now a stable version. Reports of any additional fixes needed will be appreciated!  

05-12-08
Two new features in this release.
1) The Monte Carlo module now automates the procedure of launching two or more identical batch jobs and aggregating the results after they complete. These are reported exactly as if a single large Monte Carlo experiment had been specified. If you have a dual core or quad-core machine, this effectively provides a parallel processing capability. An experiment can be run in a half or a quarter the time (depending on how many processors you have) as would be taken on a single processor machine. There is no limit to the number of simultaneous jobs, and they could be sent to other machines to extend the parallel capability indefinitely.  
2) An estimation sample must normally represent a consecutive series, without missing observations, and the specified sample is truncated to be sure missing observations are excluded. This is the appropriate behaviour for time series. For the analysis of cross-section data sets, it is now possible to specify that missing observations are simply filtered out of a specified sample, without truncating it. This means that the observations need not be consecutive, so the option is disabled if any dynamic modelling features are specified. 

22-11-08
There was a flaw in the last upload, with an incorrect file included, apologies. Only Linux users were seriously troubled, but an update is recommended to be on the safe side. Other small fixes are included.   

20-11-08
Version 4.28 allows two or more data sets to be open at the same time. This feature takes advantage of the ample memory of modern computers to allow any number of  projects to co-exist in the same session. To switch data sets, the menu command Setup / Data Sets / Switch To..., opens a popup menu with the list of sets currently in memory. Choosing a set from the list brings it to the 'foreground', while the current set takes its place in the background.  All data sets open at closedown are automatically re-opened at the next start-up, provided the disk files have not been deleted or moved. Likewise, 'exported' settings files contain all the datasets currently open.

The status bar at the bottom of the TSM window now shows the current "Data Set", instead of "Data File" as previously. Data sets are named according to the name of the file containing them, excluding extension. This means that data files must have unique names if they are to be loaded at the same time.  Files with the same name but different type, and with different storage locations (paths), are not treated as distinct. 

25-10-08
TSM Version 4.27 introduces a recoded graphical user interface for Windows, making use of Java Swing technology. Up to now, TSM has used an ingenious public domain package called JAPI (Java Application Programming Interface) due to Merten Joost (www.japi.de), adapted for the Ox language, as OxJapi, by Christine Choirat and Raffaello Seri. This code was last updated in 2003, and used the original AWT version of the Java runtime environment. It has had a few limitations such as the lack of a "table" object, tabbed dialogs, and so forth.

Tim Miller of the University of Exeter has now comprehensively revised Joost's Java code, so that the package can be implemented in Java Swing. We call the new version OxJapi2. The new package is installed under Windows by running the installation wizard in the usual way. There is one additional wizard page, where users can choose between three "look and feel" options. The appearance of the new version is similar to the old one apart from the different designs of dialog objects such as buttons and checkboxes. There is a new set of toolbar buttons, supporting the transparency attribute. The main new feature is a spreadsheet-style data editing dialog, which allows any number of series to be displayed in columns. The Windows file dialog is a lot neater and more informative, this was another weak point of the original OxJapi.  The installation program deletes any old OxJapi files and icon files before copying the new ones, so upgrading an existing installation should be seamless. So far only a Windows compilation is available, but Linux users can run TSM 4.27 with the original OxJapi code. All the required files are still available in the zip file distribution. Please see readme.txt and  Appendix A of the documentation for details.

Note 1: A number of code rewrites have been called for to make TSM work properly with Java Swing, which does a number of things differently from the old AWT Java. I hope nothing has been overlooked, but if any part of the GUI is found not to be working as it should, kindly notify me. Reversion to the old technology is not difficult, see Appendix A of the documentation. 

Note 2: We encountered a display corruption issue that will make problems for TSM 4.27, when running the JRE (V6, Update 7) with an nVidia graphics card. This appears to affect Swing applications, but not AWT. To see if you have the problem on your system, open the Java Control Panel, and then the Windows Task Manager. Put the Task Manager on top of the Java window, then give the latter window the focus (click its title bar). Check whether the Java window "bleeds through" the Task Manager window, which has the "always on top" attribute. If you get this problem, reduce the hardware acceleration of your card by 50%. (DON'T update the nVidia driver - that actually caused us more trouble!) 

17-10-08
The update posted on 14-10-08 contained a small revision to the settings load sequence, and regrettably introduced a small bug. This is fixed in today's update.

03-10-08
Two small enhancements of data analysis capability in today's release. The Hodrick-Prescott filter can now be computed for a series, in the data transformation dialog. Bivariate histograms and kernel densities for pairs of data series can be plotted in the data graphics dialog 

27-07-08
This update adds two features that may be useful primarily when working with independent samples (cross-section) data. First, a data  bootstrap is implemented with bootstrap samples drawn randomly with replacement from any designated range of the data set. Note that all the variables (dependent and exogenous) are drawn jointly in this case. (With the residual bootstrap, exogenous variables are held fixed in repeated sampling.).
Second, it is now possible to select a sample by means of indicators, so any subset of the data can be drawn, not just consecutive sequences. For example, given a dummy for female respondents in an employment survey a regression could be run on the female members of the sample alone, without requiring any sorting of the data, using a copy of the dummy series to supply the sample indicators.
One further novelty is the facility to tint the results window with your favourite colour. This might make it easier to pick out the TSM window on a crowded desktop, or maybe is just for fun. Make your selection by cycling through the Options / General. Your choice will be remembered. There are five preset shades (including the usual white) or specify your own colour by changing RGB values in the tsmgui4.h file.        

21/22-06-08
These updates have a bit more work on the simulation and bootstrap implementation of panel data models. Also bugfixes in the criterion grid plotting option. 

17/18-06-08
The version posted today is a maintenance update which tidies up the panel data implementation. It should now be possible to estimate systems of panel equations with fixed effects, by OLS or IV, just like the usual linear system case. However, systems are not yet implemented for random effects.
Another new feature (unconnected with panels) is the ability to specify models which incorporate the random disturbance nonlinearly, in simulation exercises. Previously, to be simulated an equation had to have the disturbance (which could have been Gaussian, Student-t or other) simply added on. Now it can be incorporated in other ways, or transformed, by having it appear explicitly as a variable in a coded equation. For example, one could square a Gaussian disturbance to get a chi-squared, or exponentiate to get a log-normal distribution.
This new syntax cannot be used to estimate the model, because there is no feasible way to invert an arbitrary nonlinear equation numerically. Usually, however, it will be possible to construct a complementary version of the model for estimation, using the existing "residual" format.   
 A number of bugs have been fixed, including a problem with the recently implemented Bierens specification test. It turns out that some other M-test statistics may also have been computed incorrectly in some circumstances. The effect was generally of small order, but recomputing any mission-critical M-statistics with this update is recommended.  

20-04-08
Version 4.26 introduces panel data capability. Panel data sets can now be read, manipulated and analysed, using a data file format similar (but not identical) to the Arellano-Bond DPD package. The standard estimators are implemented, that is, OLS for fixed effects and GLS/ML for random effects, and also instrumental variables.  Some basic diagnostics such as the Breusch-Pagan test and Hausman specification test are implemented, as is simulation using Gaussian or bootstrapped disturbances. Specialized GMM methods for dynamic models will, hopefully, appear in a later release.
This is quite an extensive update. It has been thoroughly tested, but there is always the chance of a problem with existing code that has had to be modified. I can see no reason not to recommend installation, but in case a problem turns up, reinstall the latest release of Version 4.25 (19-04-08). This has the graphics fixes indicated in the previous blog entry.
The main panel data operations have also been verified against the Grunfeld data set, but not all implemented features have yet been tested in all combinations. This is still work in progress. Reports of suspected issues will be gratefully received.   
 

26-03-08
An obscure Gnudraw bug has turned up involving plots of likelihood contours. Until this can be fixed, a work-around is to replace the copy of tsmgnu4.oxo in your installation with the one from Version 4.22. (See the download page for zips of old versions). Doing this will of course lose some nice graphics features that have since been added, so rename the current version of  tsmgnu4.oxo, rather than delete it, to allow reinstatement for normal use.

13-03-08
This maintenance release fixes a backwards compatibility issue with v25.29-02-2008, involving settings exported from earlier versions. The program could hang when displaying a data graphic.

Seasonal dummy generation has been enhanced so that "Quarter 1" is now the first quarter of the year, not the first observation in the file. The seasonal frequency is set automatically unless the data are undated.

07-03-08
This release has an improved installer. The Start-in directory can now be selected by typing the path or browsing the folder tree, instead of choosing from preset options. If the package is reinstalled without uninstalling first, then the current selections will be retained. making upgrading simpler than ever.   

A Gnudraw update, restoring variable font sizes, is also included.

29-02-2008
Version 4.25 has had numerous mini-upgrades in the last few weeks. Most of the changes are internal, but a few new features have been added, as follows: Data Transformation (max and min functions); formula coding (indicator functions); Summary Statistics (quantiles option); critical value look-up (Kolmogorov-Smirnov table) and most recently, improved sample selection. Now, different subsamples can be set for different program functions, such as estimation, summary statistics, log-periodogram regressions, and plotting.

On the graphics front, TSM may currently be the only package to offer colour-coded scatter plots - you can tell which part of the sample a data point belongs to by its colour; red at the start of the sample, green in the middle, blue at the end. These plots can be substantially more informative than simple scatter plots. See for example this scatter of the log(DM/$) vs. log(£/$) daily exchange rates for 1980-1996.

Regrettably, a serious bug found its way into a release posted a couple of weeks ago. This resulted in an incorrect ADF statistic being reported. Apologies for this. Fortunately, the printed value could not be mistaken for the correct one. Please upgrade your installation with the current release if you need to. 

Since TSM is work in progress, new features may not work wholly as advertised at first. They often get tweaked as more experience is gained with them. If you have trouble with any new feature, or its documentation, a report will be much appreciated.