CreditMetrics Benchmark

 

CreditMetrics™ is one of many R packages supporting the financial industry.  IBM has used CreditMetrics as a performance benchmark. It is a good choice that we emulate here.

 

CreditMetrics is portfolio credit-risk modeling framework. It was disclosed by J.P. Morgan in 1997 and subsequently spun off to RiskMetrics. The R package was contributed to CRAN by Andreas Whittmann in 2006. CreditMetrics is an example of Monte Carlo simulation. The performance of CreditMetrics primarily depends upon how fast your R distribution and hardware can perform a Cholesky factorization of a (symmetric positive definite) correlation matrix. Whittmann's implementation was based upon information in a book by Paul Glasserman.

 

REvolution speeds up CreditMetrics in two ways:

1. We have applied a performance patch to the CreditMetrics package itself. Our patch is now included in CreditMetrics version 0.0.2 on CRAN.
2. We supply multicore math libraries that CreditMetrics exploits.
   

The code we use is available allowing you to run the identical benchmark on different hardware or with different R distributions.

 

Our performance results follow:

 

	Cores	Elapsed	Parallel Efficiency	Windows Speed-Up
CRAN Vista 32-bit before patch	always 1	1727.64
CRAN with patch	always 1	129.79		13 times
REvolution R with patch compared to CRAN with patch	1	34.35		4 times
	2	27.47	61%	5 times
	4	24.85	36%	5 times
	8	23.55	21%	6 times

 

The benchmarks above were run on an Apple MacPro (Early 2008). This model features two Intel Xeon E5462 processors running at 2.8 GHz. The Windows benchmarks were run after directly booting Vista 64-bit SP1. REvolution R was version 1.2.2 which is based upon CRAN 2.7.2. The CRAN used was 2.8.1, the bleeding edge at the time. Consult REvolution's documentation to learn how to run benchmarks that use less cores than your hardware offers.