SEF-Blog: Signal Extraction and Forecasting

Real-time signalextraction is about filtering time series towards the end of a sample. It is about optimal designs of one-sided filters (some simple excel-exercises were proposed in my tutorials: 1 and 2). Ideally, the filters should

• not delay the time series (phase artifacts inherent to one-sided filters)
• Damp noise and, more generally, mask any undesirable components.

The most widely used model-based approaches rely on ARIMA-models to achieve the task (see my tutorial 1). I here briefly review structural weaknesses of this approach and propose a `creative' solution whose outcome will be a new hybrid design.  

I gave a short talk about my recent work on financial trading in our weekly colloquium-series. I thought it may be helpful to put my presentation online because it collects and summarizes results spread over the last 20 Blog-entries: you may click on the following link colloq_idp.pdf to open the document.

I'll assume that readers are `familiar' with my previous Blog-entry. There, I emphasized mean-square issues and I presented results for orthodoxe (maximum likelihood) as well as unorthodoxe (a particular DFA) estimation procedures based on proprietary R-code (available on request) as well as examples in Excel. In my second tutorial on the topic I'd like to `leave behind' the orthodoxe approach with more insistance. In particular, I'll present filter designs which

• emphasize turning points much more radically (at the expense of mean-square performances) and which
• are likely to shaken main-stream conceptions about `nice' intuitive mean-square real-time designs.

For that purpose I will rely on my Excel sample files exclusively: anyone could replicate the results in my Blog-entry perfectly (please do so and more) and all calculation steps can be deduced directly from the corresponding formulas in the spreadheet's cells.

I suggested at various places in my Blog (for example 1 and 2) that real-time detection of turning-points is a deeply counterintuitive exercise. I suggested, also, that mean-square criteria often lead to intuitively straightforward -though frequently inefficient- solutions. Some recent feedback motivated me to provide a tutorial on the topic. The empirical material has been posted in an easily accessible Excel format, see 1, but I am aware that this lose form of tutorial is not well suited for `unexperienced' users. Therefore, I here propose a step-by-step instructions manual intended for the unexperienced among us. Experts are welcome as well, particularly maximum likelihood aficionados. 

The following series of exercises in Excel intends to illustrate

• Misspecification issues
• `Interpretability' of mean-square (model-based) solutions
• Inefficiency of model-based approaches with respect to turning-point detection
• The complexity of the structure of real-time estimation problems
• The deeply counterintuitive structure of turning-point problems and the particular gestalt of `improved solutions'.

Part I is devoted to the mean-square error norm. I distinguish two approaches:

• The orthodoxe: the traditional maximum likelihood (ARIMA) approach as implemented, for example, in X-12-ARIMA or TRAMO.
• An unorthodoxe: a particular version of the `direct' filter approach (DFA)

My experience along a pretty eclectic range of practical prospective estimation problems (forecasting,/filtering) is that mean-square error/estimation criteria are handy and well-suited for lazy econometric minds because solutions are often intuitively interpretable, at least a posteriori (which is all you need for autosuggestive peace of mind). Unfortunately, many users simply don't care about `mean-square' performances and elegant solutions. Instead they want to `see' turning points before competitors or rivals  can adjust or react. Do you really believe that traditional econometrics/time series analysis maximize this kind of likelihood? 

I have prepared illustrative examples in an universal Excel framework - this cheap and dirty banker's tool is a formidable didactical tool too - for a INSEE workshop in Paris, see this previous Blog-entry (the introductory French part of the text can be ignored). I would like to encourage anybody unaware of timeliness and reliability issues (I don't mean `job-attitudes' but `real-time signalextraction') to perform some simple exercises based on the proposed Excel files. The effect is similar to Yoga, particularly the ARMA(2,2)-ZPC examples: it detracts from main-stream beliefs and focusses on statistical `essence'. In the following Blog-entries (see 0) I'll propose a tutorial for `unexperienced' users. Experts are welcome.

In a previous Blog-entry (Why do Traders Often Consider Crossings of Trading Filter-Pairs?) I emphasized the importance of designing real-time bandpass filters more effectively than by relying on (traditional) crossings of MA-filters. In a following entry (Efficient Trading Filters: Flexible Positioning of the Pass-Band, Strong High-Frequency Smoothing, Small Time-Shifts) I emphasized the inappropriateness of the traditional econometric approach, based on time-domain maximum likelihood methods. My purpose here is to illustrate briefly the basic ideas behind the customized optimization criteria of MDFA-XT, our experimental real-time trading filter for the MSCI emerging markets. Then I'd like to disclose vital characteristics of the real-time filters: this is something that maximum likelihood aficionados typically and superbly ignore. Finally, I'll reveal extracted components (extracted from the MSCI and the BRIC-indices) and suggest that our multivariate filter generalizes dynamic factor models by relaxing unnecessarily severe restrictions of the latter approach.  

I'm very happy to get the opportunity to present our new real-time trading algorithm MDFA-XT at this year's R-Finance Conference in Chicago. I look forward for feedback and discussions in this exciting finance-round.