The motivation for building the Disruptor was the search for the most efficient ways to pass messages from one thread to another. That, in turn, requires some synchronisation between the threads: the producing thread must be able to inform the consumer that the next message is ready to be processed. In [1], the authors experiment with different ways of synchronising threads in Java. I decided to perform similar tests in C++ to see if there is a substantial language dependency in the results.

The table below shows average time, in milliseconds, to perform 500 million increments of a 64-bit unsigned integer. The results were produced on an Intel Core i3-2310M CPU (2 physical cores with hyperthreading give 4 logical cores) running at 2.10GHz. Of course, the results for other CPUs might differ greatly, so it is the ratio between the execution times that matters, not the times themselves. The last column shows the ratio of each measurement to that of an un-contended read and write.

Method | Time, ms | Ratio |

Atomic read and write, no contention | 1,774 | 1.00 |

Atomic read and write, contention | 1,863 | 1.05 |

Atomic fetch_add, no contention | 5,539 | 3.12 |

Atomic fetch_add, contention | 30,264 | 17.05 |

Locking a mutex, no contention | 17,729 | 9.99 |

Locking a mutex, contention | 153,566 | 86.56 |

In the last four rows, contention means that two threads increment the same variable. For atomic reads and writes, however, an experiment with two threads updating the same variable would make little sense, as no correct real-life program would do that. In this case, contention means that two threads update different variables which are located next to each other in memory, hence they probably will be accessed through the same cache line.

These results are not very different from the results shown in section 3.1 of [1], apart from one. The first two rows are almost identical. This means that the speed of atomic reads and writes does not differ much if the variables written by the two threads are next to each other or not. In other words, the effect of “false sharing” (section 3.4 of [1]) has almost no effect on the results of this experiment. This detail, however, is not relevant to the conclusion of the experiment: traditional ways of synchronisation, such as mutex locking or even atomic fetch_add, are far inferior to the algorithms where each variable is modified by only one thread (although it can be read by many).

The code which I used to run the experiments can be downloaded as timing.tgz. I compiled it with GCC version 4.8.2 and ran it on 64-bit Ubuntu 14.04. I did not try, but it should be possible to compile the code with any C++11-compliant compiler.

The tar includes the following files:

timing.h | Utilities to measure execution time. |

atomic_update.cpp | Measurement of atomic reads and writes. |

fetch_add.cpp | Measurement of fetch_add. |

locked_update.cpp | Measurement of locking a mutex. |

mean_stddev.cpp | Utility to compute the mean and standard deviation. |

run.sh | Script that compiles the code and runs the measurements. |

Almost all code is self-explanatory. Each measurement is run a few times (from 2000 to 200, depending on what kind of measurement it is), and the average running time is computed. Besides, the standard deviation of results is computed. A high standard deviation shows that the conditions were not consistent throughout the experiment, which means that the results might be off. In my case the standard deviation was never greater than 60, which is fairly decent.

The last file on the list, run.sh, is probably the only file that would need modification for other compilers and operating systems.

I am curious what the results might look like for other compilers and OS’es, so if you decide to run the measurements, I would appreciate if you share the results with me. The measurement code is free to use and modify, it is provided “as is”. Keep in mind, though, that it might take a fairly long time to run the measurements, even on modern hardware; I usually do it overnight.

**References**

[1] http://disruptor.googlecode.com/files/Disruptor-1.0.pdf

[2] http://martinfowler.com/articles/lmax.html

[3] http://mechanitis.blogspot.nl/2011/06/dissecting-disruptor-whats-so-special.html

[4] http://mechanitis.blogspot.nl/2011/07/dissecting-disruptor-why-its-so-fast.html

The movie business is risky, but the returns can be high. Better yet, the movie industry performance is only loosely correlated to that of the rest of the economy, which makes it an attractive diversification option.

In the Netherlands, investing in motion picture production is especially good because of the favourable tax regulation. To put it simply, if the movie flops, the losses of the private investors are deduced from the tax they have to pay in that year. The production company usually agrees that beforehand with the tax office. Thanks to that, potential loss of a private investor is limited to a small fraction of the investment, while potential returns are high.

Recently I got an offer to invest in a new movie. However attractive this seems at the first sight, I need to do my homework before deciding to invest. I have to calculate the expected return and variance and compare these to other investment options. The movie production company published the projected returns contingent on the movie’s gross box office earnings in the Netherlands, so if I can estimate the box office, I will know the distribution of the investment’s returns. In this blog post I will analyse the historical distribution of the movies’ earnings.

Existing research [1] shows that the gross earnings of the movies released in the US each year follow a power law distribution with Pareto exponent . To estimate the box office of a Dutch movie, I will need the statistics of the movies’ earnings in the Netherlands.

If I had limited the statistics to the Dutch-language movies released in a single year, I would have got hopelessly little data. To remedy the situation, I assumed that

- The distribution of movies earnings is the same every year.
- Movies have equal chances of success regardless of the language.

I do not have any evidence to support these claims, but they sound reasonable. Besides, I cannot get much further without any of them. Together, they allow me to use the complete movie-going statistics of several years, which provides a sufficient number of data points.

Ticket prices change, so instead of looking at gross earnings, I considered the number of tickets sold; this would give equal weight to each year’s results. Luckily, the ticket sales data for the top 1000 movies released from 1991 to 2012 are available from the Dutch Film Distributors’ Association (NVF); the pdf can be downloaded here.

As I was looking for a power law distribution, I plotted the logarithm of the film’s rank versus the logarithm of the number of sold tickets. The data (blue dots) are shown on Figure 1 below.

I expected the dots to lie close to a straight line; that is what a power law distribution looks like on a log-log plot. The best linear fit to the data is shown by the black line. This fit is rather poor; . The Pareto exponent implied by the linear fit is .

Surprisingly, a quadratic fit (red line) is very good; . This contradicts the findings in [1], but the data points lie remarkably close to a parabola! I have no explanation for this; I would be curious to know if anyone got similar results from the analysis of the movie earnings distribution elsewhere.

Finally, I have to reiterate that the fit was done for the top 1000 movies released from 1991 to 2012. According to the data published on boxofficenl.net, the total of 5846 movies were released in the Netherlands during these 22 years. The top thousand represents of all released movies.

**References**

- Blockbusters, Bombs and Sleepers: The income distribution of movies. Sitabhra Sinha and Raj Kumar Pan, 2005. Available at http://arxiv.org/abs/physics/0504198.

**Disclaimer**

This is not an offer, solicitation or advice to invest. Past performance offers no guarantee for the future.

]]>By all means there can be reasons to stick to good old measures. One is convenience. It might be that the usual SI units are inconvenient for being used in finance. It took me some time to come up with an example of a SI measure that is utterly unfit for everyday use in our trade.

Consider interest rate. It has the dimension of frequency (1/time). Let me see if the SI unit of frequency, hertz (Hz), is convenient for expressing commonly used rates.

Take the interest rate of 1% per annum, simply compounded, quoted according to the Actual/365 day counting convention. How much is it in SI?

1 year = 365 days (by the day count convention) = 365*24*3600 seconds.

Imagine yourself quoting a rate of .317 nanohertz to a customer.

Originally I planned to post this on 1 April, but something distracted me… Late is better than never, anyway.

]]>https://bitcointalk.org/index.php?PHPSESSID=309cf9027c5144b5cf3eef31f09866d0&topic=35812.0

]]>

**Abstract**

In this short how-to we will use Monte Carlo simulation to compute CVA. For simplicity, we will ignore the possibility of our own default (i.e. we will calculate unilateral CVA) and will not take wrong-way risk into account.

**Introduction**

Credit value adjustment (CVA for short) is the difference between the price of a derivative without counterparty risk and the price of the same derivative adjusted for counterparty risk. If we have more than one trade with a certain counterparty, we usually cannot correctly calculate CVA per trade, but we can always calculate the total CVA for the counterparty.

According to [1], unilateral CVA for a counterparty (ignoring wrong-way risk) is

(1)

where is the loss given default, is the risk-free discount factor for , is the risk-neutral expectation of our exposure to that counterparty at time .

Exposure is the amount of money that the counterparty owes us if it defaults at time .

Suppose we have trades with the counterparty. If all of these trades are covered by a netting agreement, our exposure to this counterparty at time is

, where is the MtM value of the i-th trade at time .

If there is no netting agreement, our exposure is .

At any rate, our exposure is a random variable, and we need to know its expected value for each . For , is equal to the current exposure .

In practice, we do not calculate for each , but:

- Select k points .
- Calculate .
- Compute integral (1) as the sum

(2)

Here, is the probability that the conterparty defaults between and . Probabilities of default can be obtained from internal ratings or CDS spreads. The simplest (but not the most accurate) technique is given in [3], paragraph 98:

where is the CDS spread at time .

**Sample portfolio**

Our sample portfolio consists of a single trade: an equity forward. At time we buy from the counterparty 1 share of stock at the price . By contrast to an option, in an equity forward tansaction we have an *obligation* to buy the stock, even if at time price is higher than the market price of the stock. In our example, stock does not pay dividends.

The value of our trade at time is

where is the price of stock at time .

**Model**

We take a simple but popular model of the stock price evolution: geometric Brownian motion. Let be the stock price at time t, then we postulate that is the solution of

(3)

where is Brownian motion, and are parameters of the model.

In addition, we make a fairly unrealistic assumption that the interest rate is zero and will remain zero until time . Although this makes our results unusable in real life, our example will, without additional complication, demonstrate the application of Monte Carlo simulation to calculating CVA.

**Calibration of the model**

Parameter in our model corresponds to the *volatility* of stock . We take as the volatility implied by the prices of options on that expire at . To make our model risk-neutral, we take .

**Computing expected exposure**

We are now ready to calculate expected exposures. Remember that we need to do that at points in the future, . For each of those points, the algorithm is the same, so we consider the computation for some point .

Monte Carlo method is a way of computing integrals (see [2] for a detailed treatment of this method). Expected value of a random variable is an integral, so we can employ Monte Carlo to calculate the expectation. To be fair we note that Monte Carlo is beneficial when the integral in question is multidimensional (that is, the number of dimensions is greater than 2). Our example is single-dimensional, but we will use Monte Carlo anyway.

The solution to equation (3) is

where is today’s price of the stock, is a standard normal random value.

We begin our first Monte Carlo iteration by drawing random number from the standard normal distribution and compute the first sample value using calibrated values of and :

Next we calculate the exposure in this scenario:

Thus we have created one random scenario (by drawing ) and calculated the exposure value for this scenario. This concludes the first Monte Carlo iteration.

To obtain the expected exposure at , we perform many (say, a thousand) Monte Carlo iterations and obtain exposure values for each scenario: , where is the number of iterations. Our estimate of the expected exposure at time is the average of all sample exposures:

**Putting it al together**

Having computed expected exposures , we can calculate CVA by formula (2).

**References**

- Zhu, Steven H. and Pykhtin, Michael, A Guide to Modeling Counterparty Credit Risk. GARP Risk Review, July/August 2007. Available at SSRN: http://ssrn.com/abstract=1032522
- Glasserman, Paul, Monte Carlo Methods in Financial Engineering (Stochastic Modelling and Applied Probability), Springer, August 2003.
- Basel Committee on Banking Supervision, Basel III: A global regulatory framework for more resilient banks and banking systems. Bank for international settlements, December 2010 (rev June 2011). Available at: http://www.bis.org/publ/bcbs189.pdf

Here is the blog post about it that I wrote for the Global Risk community.

]]>

The EU Court of Justice has ruled that one can’t copyright a computer program’s functionality. Further, they state: “The purchaser of a licence for a program is entitled, as a rule, to observe, study or test its functioning so as to determine the ideas and principles which underlie that program.”

Here is the article on Groklaw:

http://www.groklaw.net/article.php?story=20120502083035371

]]>

The outsourcing company usually claims that keeping their own IT support team would be more expensive than outsourcing. I believe this is a fallacy, and here is why.

Firstly, consider an organisation (let’s say it’s a bank) that has its own IT support team. This team is shown on Figure 1.

Say, the team consists of 20 people, which is just enough to keep the IT infrastructure up and running. Obviously, his team needs a manager. The manager is responsible for task allocation, making sure that not all of the team take holidays at the same time, conducting end of year performance reviews — in short, the manager does not do anything useful for the bank’s IT systems. The manager represents the overhead which is needed for the team to work well. In other words, the bank pays salaries to 21 people, but only 20 of them do some useful work.

Next, let us consider the case where the bank has outsourced the IT support to another company, the solution provider. The solution provider is a big company that serves many different clients. Suppose our bank still needs 20 people to maintain its IT infrastructure. Like before, these 20 people need a manager. However, the solution provider is a big company that consists of many such teams. Their managers, in turn, have higher-level manager, that oversees multiple teams. These higher-level managers report to other, even higher, managers, and so on until the pyramid converges to one person, the president of the company. This is shown on Figure 2.

Here, the bank that uses the services of the 20 people, pays not only their and their manager’s salaries, but also a fraction of the salaries of all the higher management up to the top (otherwise, where would their salaries come from?)

Paying just a bit more for the same amount of work would not be such a big deal, but it does not stop here. There is one more figure that stands between the bank and the solution provider: the account manager. The job of the account manager is to make the bank pay as much as possible for the services it receives. Guess where the account manager’s salary comes from? Also from the fees that the bank pays to the solution provider. The bank, in fact, feeds its worst enemy.

But wait, before the bank decided to outsource its IT support, they made calculations which showed that the fees of the solution provider are lower than the costs of having IT support inside the company. How did they do these calculations? The trick is to design outsourcing such that the solution provider will have to do less work than an internal support team would have done. Let us look at an example.

One morning at the bank that I am consulting, my Microsoft Access application crashed and damaged the database file. “No big deal,” thought I, “the files are on a network server, which is backed up nightly. I can ask for the last night’s backup.” It should take no longer than an hour to restore a file from a backup, right?

I phoned the bank’s IT helpdesk and explained the problem. “Ok,” said the person on the other end of the line, “I will make a ticket for the backup/restore team. You can expect the problem to be solved within 24 hours.” “Can it be done quicker?” asked I anxiously. “I need this file to complete an urgent task.” “All I can do is make a ticket,” answered the helpdesk person, “the rest is up to the backup and restore team, and they have 24 hours, according to the SLA.”

The bank has an agreement (Service Level Agreement, or SLA) with the solution provider, which specifies the time allotted to solving IT problems. As long as the solution provider conforms to the SLA, they have little incentive to hurry up with doing their job.

Bottomline: outsourcing IT support looks attractive on paper, but in reality the bank gets much less work done for the money they pay.

I am not trying to say that outsourcing is generally a bad idea. In many cases it is exactly the thing to do. Every two years, the banks needs to repaint the facade of its office building. It hires services of a painting company. The painting company does the job in a week and paint some other facades before it is time to repaint this bank again. There is no reason to keep the painters on the bank’s payroll.

Now let us look at three more arguments that are often used to justify outsourcing, and see why they do not apply to IT support.

1) Doing IT support is not the bank’s core business.

This argument would only work if one believes that a

company doing only its core business is good for the company. This is simply not true. Say, the core business of a railroad is to bring passengers from one place to another. If the railroad decides not to sell hotogs on the platforms because “selling hotdogs is not our core business”, someone else will do it and take all the money.

2) IT solution providers have more expertise in IT support than the bank.

This is not true either. They just hire anybody who is willing to work for the salary they pay, train them quickly and send them to the customers. When the bank owns its IT support, it have much more control of the quality of its workforce.

3) Outsourced work can be done in other countries where wages are lower.

This was true several years ago, but by now the people who can do the work became expensive regardless of their geografical location. The people that remained cheap are probably not a good hire even for a low salary. Besides, IT support requires the engineer’s physical presence at the customer’s site so often that one has to keep a fairly big team co-located with the customer anyway.

So, why do so many companies outsource their IT support? The reasons are plentiful, for instance:

1) It looks good on paper. The manager who has outsourced IT support can claim that he has saved a lot of money for the company.

2) It shifts the responsibility to the solution provider. If an internal IT support group messes things up (people do make mistakes every now and then), their manager gets the blame. If an external solution provider messes things up, they are to blame, not the manager who outsourced the work to them.

3) Due to the herd instinct. Outsourcing is popular, so people tend to do this because everyone else does. 20 years ago a vast majority of males smoked for exactly the same reason.

Sadly, many companies continue to outsource their IT support, and are likely to continue to do so in the future.

]]>