Vi Users Have More Control

I was testing the macro focus on my new camera this morning, and thought this might be amusing to the programmers out there. So, at the risk of starting a text editor flame war...

Alert! Testing Javascript's alert function with Jasmine

I was refactoring some javascript code at work the other day. Literally, 3000 lines of logic to decide on whether to throw an alert and what to include in the message, if so. I kid you not. There's a serious problem with some of the legacy code where I work, though, admittedly, the logic was a big tricky. All but about 300 lines of that code was pure duplication (copied, more or less line for line across 10 pages). So, by the end of the day I'd reduced the 3000 lines to about 60, and added about 240 lines of Jasmine tests (still 300 lines, but now heavily tested).

Anyway, in doing this, I came up with a way to test the function which threw the potential alert messages. I thought I'd share this idea, as I rather enjoyed coding it. Either way, I wanted to document it for my own reference, and hopefully others will find it useful, too.

The gist of the problem here was that there was a single function call (triggered from clicking a 'Save' button on various pages), which susequently called a number of other functions, depending on the case being tested, each of which had the potential to add something to the alert message. I had refactored these subsequent functions to just return strings with their various contributions, and so those functions were very easy to test individually.

What was tricky, was figuring out how to test the original function that actually throws the alert, depending on the responses. How do you test an alert? I guess there are various ways, but here's what I came up with...

Imagine if you will, the 'main' function is called 'alert_me' which ultimately just returns so that the form post can be submitted. 'alert_me' calls a number of other functions, all of which use a function called 'get_val' to reference the data that they use to calculate their response (I tried to simplify this problem for this post as best I could, so we could just focus on the problem at hand. Hopefully, this makes sense.)

I set up an object to contain default values for the various possible responses of 'get_val' (for this exercise foo, bar, or baz), and a helper function to set them ('set_val'). (Note: I'm not testing 'get_val', but I need to be able to mock its output in order to get the correct response from 'alert_me'). I also use a variable to capture the alert message which I can set by overriding javascript's 'alert' function.

 describe("alert_me", function() {  
   var get_val_rtn_val, alert_msg;  
   var set_val = function(field, val){ get_val_rtn_val[field] = val; };  
   beforeEach(function() {  
     get_val_rtn_val = { // reset values  
       'foo': 0,  
       'bar': 0,  
       'baz': 0  
     };  
     spyOn(window, 'get_val').andCallFake(function(field) {  
       return get_val_rtn_val[field];  
     });  
     alert_msg = '_default_';  
     spyOn(window, 'alert').andCallFake(function(msg) {  
       alert_msg = msg;  
     });  
   });  
   it("tests the test", function(){  
     expect(get_val('foo')).toEqual(0);  
     set_val('foo', 1);  
     expect(get_val('foo')).toEqual(1);  
     expect(alert_msg).toEqual('_default_');  
     alert('foobar');  
     expect(alert_msg).toEqual('foobar');  
   });  
   it("contains '_default_' for default case", function(){  
     alert_me();  
     expect(alert_msg).toEqual('_default_');  
   });  
   it("contains 'Wheezy' for foo=1", function(){  
     set_val('foo', 1);  
     alert_me();  
     expect(alert_msg).toEqual('Please note the following: Wheezy');  
   });  
   it("contains 'Dopey' for bar=1, baz=1", function(){  
     set_val('bar', 1);  
     set_val('baz', 1);  
     alert_me();  
     expect(alert_msg).toEqual('Please note the following: Dopey');  
   });  
   it("contains 'Wheezy' and 'Dopey' for foo=1, baz=1", function(){  
     set_val('foo', 1);  
     set_val('baz', 1);  
     alert_me();  
     expect(alert_msg).toEqual('Please note the following: Wheezy, Dopey');  
   });  
 });  

Naturally, if 'alert_me' is working as expected, these tests should pass!

I'd love to hear feedback about this solution. It seemed pretty simple and straightforward to me. I find Jasmine to be flexible and fun to work with.

As a side note, one of the things that I really enjoyed about writing all these tests was that it made me think differently about the way I write Javascript. I believe the refactored code was a lot more elegant, largely due to the fact that it was designed to be easily tested.

A First Look at node.js, A Logging Application

I had a chance this week to take a first look at node.js. I'd been meaning to do this at some point, as it's a critical component in my ongoing investigation of full-stack JavaScript, so I was glad to get this opportunity.

I've already covered a bit about MongoDB, but, obviously there's a lot more to go over here. One thing that is worth noting is that Mongo seems to be emerging as the persistent database of choice in the full-stack frameworks (with Redis often being used for caching). [Just FYI, I'll be at the Mongo Meetup in Boston this coming Wednesday]

But, while one has many choices on the datastore side of things, there's one technology that seems to be indispensable for doing full-stack JavaScript, and server-side JavaScript in particular, and that is node.js. In short, Node is a JavaScript engine, based on Google's super-fast and open source V8, with particular emphasis on network application capabilities, CommonJS library implementation, and package management (via npm). Although it's possible to fork threads, and there's even a cluster API, a node application is generally designed to work in a single event loop, using non-blocking, asynchronous, I/O to achieve great efficiency in processing requests.

Point is, it's fast.

Nearly as fast as Java, and orders of magnitude faster than Ruby.

It's "event-driven," using callback functions to deliver asynchronous functionality, very much like the way JavaScript works in the browser. However, it's a very different way of thinking when programming server-side code. My first pass at understanding it took a little effort. I focused on using the setTimeout function in JavaScript to test some assumptions about how events are dealt with. Initially, I was quite confused because I would trigger two requests in short succession in two Firefox tabs, each of which would fire a 10 second wait using setTimeout, and the requests were served serially, one after the other, taking a total of 20 seconds to process both. However, as it turns out, this was happening because of the way Firefox was managing the requests. Using curl on the command line and executing multiple requests more or less simultaneously revealed that these "waits" are, in fact, running in parallel.

This first attempt can be seen here:

 var http = require("http");  
 function do_thing(response, callback){  
  var d=new Date();  
  console.log('calling setTimeout '+d.toLocaleTimeString());  
  var timer = setTimeout(function(){  
   var d=new Date();  
   console.log('timeout callback '+d.toLocaleTimeString());  
   respond(response);  
  }, 10000);  
  var d2=new Date();  
  console.log('setTimeout called '+d2.toLocaleTimeString());  
 }  
 function respond(response){  
  var d=new Date();  
  console.log('sending response '+d.toLocaleTimeString());  
  response.writeHead(200, {"Content-Type": "text/plain"});  
  response.write("hello at "+d.toLocaleTimeString());  
  response.end();  
 }  
 http.createServer(function(request, response) {  
  var d=new Date();  
  console.log('got request '+d.toLocaleTimeString());  
  do_thing(response, respond);  
 }).listen(8888);  
 var d=new Date();  
 console.log('running... '+d.toLocaleTimeString());  
 // UPDATE: Firing two requests curl in two Terminal windows resolves the issue below  
 // This code is good. The test (using Firefox) was bad.  
 //Two web requests in close succession, still have to wait 20 secs for 2nd request to complete...  
 //OUTPUT AS FOLLOWS:  
 //  
 //running... 12:12:00  
 //got request 12:12:06  
 //calling setTimeout 12:12:06  
 //setTimeout called 12:12:06  
 //timeout callback 12:12:16  
 //sending response 12:12:16  
 //got request 12:12:16  
 //calling setTimeout 12:12:16  
 //setTimeout called 12:12:16  
 //timeout callback 12:12:26  
 //sending response 12:12:26  

So, the next order of business was to use this knowledge to implement an actual application. The task at hand, a logging server. The basic requirement is this: HTTP POST with some arbitrary data, writes to Mongo database. Any application (client) can post data, in a format of it's own choosing (given a few required fields so that the source could be identified and filtering done). This was essentially fulfilling an "audit" requirement (details unimportant). This is about as simple as it gets, folks, for a real-life application.

I initially looked at Mongoose for interacting with the database. It's a pretty clean interface, and worked well. However, soon after getting that more or less working, I came across this module (which is part of the flatiron framework), a "multi-transport" logging library. This made sense, to use a logging library to do logging. The ability to use various storage mechanisms is very nice, and the fact that it supported Mongo out of the box was great.

One thing to note about Mongoose, in case this helps anyone who's looking at it, is that it uses the concept of a Schema. Which a first blush seems kind of silly when dealing with Mongo (which is schemaless, duh). And, in particular, I wanted to have a schemaless "logentry" which could be formatted by the application submitting the data. Fortunately, Mongoose supports a datatype called "Mixed" which essentially let's you pass a Hash, like so:

mongoose.model('AuditLog', new Schema({ application_name: String, log_entry: Schema.Types.Mixed, timestamp: Date }));

In any case, Mongoose was not part of the final solution. Winston was.

There's one other important piece which came up, though, too. And that was the issue of "frameworks." Flatiron, for example, where Winston comes from, is a framework. A bit of Googling, though, and I decided that the framework to use was express. It seems to be the node framework with the most traction. It also is very lightweight, and is the framework upon which most of the more fully-formed frameworks seem to be built (i.e. Tower.js and Derby, both of which I may try to look at for future applications).

Express certainly helped dealing with posted data and routing, which otherwise could have been tricky.

Putting all the pieces together, one gets this (complete with URL-based versioning which I chose to implement despite the controversy on this subject):

 // requirements  
 var winston = require('winston');  
 var MongoDB = require('winston-mongodb').MongoDB;  
 var express = require('express');  
 // set up winston  
 winston.add(MongoDB, {db:'winston'});  
 winston.remove(winston.transports.Console);  
 winston.emitErrs = true;  
 winston.handleExceptions(new winston.transports.File({ filename: 'exceptions.log' }))  
 // set up express app  
 var app = express();  
 // found out the hard way that this middleware is required to parse data  
 app.use(express.bodyParser());  
 // routing  
 // note: urls not implemented will serve 404 not found  
 app.all('/api/log', function(req, res){  
  res.status(301).redirect('/api/v1/log');  
 });  
 app.get('/api/v1/log', function(req, res){  
  res.send(500, { error: 'GET not implemented in v1' });  
 });  
 app.post('/api/v1/log', function(req, res){  
  var required = ['appname', 'logname', 'source', 'logentry'];  
  for(param in required){  
   if(! req.param(required[param])){  
    res.send(500, { error: required[param]+' is required' });  
    return false;  
   }  
  }  
  winston.info('Winston Logging', req.body);  
  res.send('Thanks for that, '+req.body.appname);  
 });  
 // start server  
 app.listen(8888);  
 console.log('Listening on port 8888');  

This must seem incredibly naive, and, indeed, it is. However, it gets the job done. Future plans include responding to GET request to return log data (for querying).

Now, one final note: testing. Oh, jeez. This was not a simple thing.

It's not clear to me that this aspect of node development is entirely full-formed. I didn't have a lot of luck finding consensus in this area. I chose, in the end, to use Vows. I'm not entirely certain I got this right, and I plan to spend some time fleshing this out at a later date. Problems with this solution, as it currently stands, are: environment management (i.e. write test data to production db?), and the fact that I have to have the Mongo and Node servers running in order to run the tests. Not to mention the fact that I'm not testing the asynchronous code, i.e. callbacks, although this seems like it is possible, I just haven't figured it out yet. I believe there are probably better ways of going about testing a node app, but I am a novice, and have much to learn.

For the record, though, here's what I've got for tests:

 var vows = require('vows'),  
  assert = require('assert'),  
  request = require('request'),  
  apiUrl = 'http://127.0.0.1:8888';  
 var apiTest = {  
  general: function( method, url, data, callback ){  
   //console.log(data);  
   request(  
    {  
    method: method,  
    url: apiUrl+(url||''),  
    followRedirect: false,  
    json: data || {}  
    },  
    function(req, res){  
     callback( res );  
    }  
   )},  
  get: function( url, data, cb ){ apiTest.general( 'GET', url, data, cb ) },  
  post: function( url, data, cb ){ apiTest.general( 'POST', url, data, cb  ) }  
 }  
 function assertStatus(code) {  
  return function (res, b, c) {  
   assert.equal(res.statusCode, code);  
  };  
 }  
 var log_entry = { foo: 'bar', baz: 'blah' };  
 var log_good = { logentry: log_entry, appname: 'my appname', logname: 'my logname', source: 'my source' };  
 var log_no_appname = { logentry: log_entry, logname: 'my logname', source: 'my source' };  
 var log_no_logname = { logentry: log_entry, appname: 'my appname', source: 'my source' };  
 var log_no_source = { logentry: log_entry, appname: 'my appname', logname: 'my logname' };  
 var log_no_logentry = { appname: 'my appname', logname: 'my logname', source: 'my source' };  
 // Create a Test Suite  
 vows.describe('Server').addBatch({  
  'when doing get request on root url': {  
   topic: function () { apiTest.get('', log_good, this.callback ) },  
   'returns error': assertStatus(404)  
  },  
  'when doing get request with badly formatted url': {  
   topic: function () { apiTest.get('/foo', log_good, this.callback ) },  
   'returns error': assertStatus(404)  
  },  
  'when doing get request with default url': {  
   topic: function () { apiTest.get('/api/log', log_good, this.callback ) },  
   'returns redirect': assertStatus(302) // note: this actually is a 301, but the test seems to need 302  
  },  
  'when doing post with incorrect data: no appname': {  
   topic: function () { apiTest.post('/api/v1/log', log_no_appname, this.callback ) },  
   'returns error': assertStatus(500)  
  },  
  'when doing post with incorrect data: no logname': {  
   topic: function () { apiTest.post('/api/v1/log', log_no_logname, this.callback ) },  
   'returns error': assertStatus(500)  
  },  
  'when doing post with incorrect data: no source': {  
   topic: function () { apiTest.post('/api/v1/log', log_no_source, this.callback ) },  
   'returns error': assertStatus(500)  
  },  
  'when doing post with incorrect data: no logentry': {  
   topic: function () { apiTest.post('/api/v1/log', log_no_logentry, this.callback ) },  
   'returns error': assertStatus(500)  
  },  
  'when doing post with correct data': {  
   topic: function () { apiTest.post('/api/v1/log', log_good, this.callback ) },  
   'is success': assertStatus(200)  
  }  
 }).run(); // Run it  

And that's it! (Other than the package.json file which handles dependency management via npm, similar to the way that a Gemfile and bundle does in Rails.)

Not much more to say, really, other than that this was a fun and interesting first crack at node.js, which I hope to get an opportunity to work more with in future.

I look forward to continuing to get to grips with full-stack JavaScript, and I hope to be able to implement a "real" application soon, and will blog about it here. In the meantime, and, in lieu of gifts, please enjoy this comic.

Mongo 2.2 Aggregation Framework: Stock Price Data

Just wanted to give a shout out to the MongoDB (http://www.mongodb.org) folks for the new Aggregation Framework in version 2.2 (which thankfully was officially marked as stable about three weeks ago).

I've been using Mongo to build an ad-hoc reporting (real-time basic statistics with arbitrary filters, etc) app at work, and had previously been using Map Reduce to do realtime data aggregation, much to my pain and anguish. To be fair, Map Reduce was not designed to work in real-time, but I found that it worked well enough on small datasets. When the data numbered in the tens of thousands, however, speed became a real problem.

The nice thing about Map Reduce is that it all takes place in the database. You can just read your results from your target collection when the computation is done. Mechanically, this is a great way to go. But if you simply need your computations to happen in real-time, then you're stuck with bad performance.

Benchmarking (at least for this application) revealed about a five times speed improvement using the Aggregation Framework. This meant that operations on datasets of 10-20K records, doing calculations on 100-200 values (so, getting into the millions of data points) could actually happen in real-time. A late night of code refactoring, and problem solved. [Note: in case you're wondering how to get 'round the 16MB doc size limit for this, I set up "batch processing" of values, so, for example in Ruby: variable_list.each_slice(10){ ... }].

One thing that's quite different about using this new framework, is that the computation happens in the application code (in this case Ruby). The aggregation operations are only being used to extract the data for processing. With Map Reduce, you can write arbitrarily complex functions to operate on your data. With the Aggregation Framework, this has to happen in your application (to be overly simplistic, think of aggregation as the map stage, while the reduce/finalize is done in the client -- at least that's true in our particular circumstance, as we have to operate on the data in ways that the Aggregation Framework itself does not allow). So, no doubt, in a year or two (as our database grows), we will have to look into other mechanisms for optimization. But this is great for now.

I wanted to do a quick demo of the new features, so I turned to my stock price data collection. Basically, I have a script that's been collecting intra-day trading info on the Dow Jones stocks from Yahoo Finance. It's been running about over a year and a half, so there's a lot of data there, and it's quite useful for exercises such as this.

The scripts I used for this demo can be found on github (unfortunately, the data itself is quite large, so I did not include it this time): https://github.com/sqpierce/stocks

Here's a rundown of the components:

• Firstly, there's a script to grab the data (this is being run by cron each minute):
• Second, there's a Ruby script I just wrote to convert all the data files into a single JSON file for importing into Mongo
• Last, here are some basic Aggregation Framework operations, just written in JavaScript and executed via the Mongo shell (for now)

SHELL SCRIPT TO GET DATA

 #!/bin/bash  
 # has been running approx 1.5 yrs every minute while NYSE is open  
 DIR=~/projects/stocks/data  
 hour=`date -u +%H`  
 day=`date -u +%u`  
 file=$DIR/`date -u +%F`  
 # Dow Jones stock symbols  
 syms="MMM+AA+AXP+T+BAC+BA+CAT+CVX+KO+CSCO+DIS+DD+XOM+GE+HPQ+HD+IBM+INTC+JNJ+JPM+KFT+MCD+MRK+MSFT+PFE+PG+TRV+UTX+VZ+WMT"  
 # Yahoo finance stock format: http://dirk.eddelbuettel.com/code/yahooquote.html  
 #symbol, name, last trade date, last trade time, last trade price, dividend yield, P/E ratio, volume, day's value change, previous close  
 fmt="snd1t1l1yrvw1p"  
 url="http://download.finance.yahoo.com/d/quotes.csv?s="$syms"&f="$fmt  
 log (){  
  echo "#("`date -u`")" $1 >>$file  
 }  
 dostock (){  
  log "file "$file" hour "$hour" day "$day" dir "$DIR  
  log "using command: curl $url"  
  curl $url >>$file  
 }  
 # note that $hour throws error when 08 or 09 because interpreted as octal - need to investigate  
 # just changing to 8 or 9 as string using below doesn't seem to fix problem (or, rather, creates another problem  
 # hour=`date -u +%H`|sed 's/^0*//'  
 # doesn't matter as we're using UTC, so those hours are out of range  
 if (( $hour > 12 && $hour < 23 && $day < 6 ))  
 then  
  dostock  
 fi  

RUBY SCRIPT TO MUNGE DATA

 # utility to munge data into single file for insertion into Mongo  
 # ruby to_json.rb > stock_data.json 2> to_json.log  
 require 'csv'  
 DEBUG=false  
 dir='data'  
 count = 0  
 Dir.entries(dir).each do |file|  
  next if file =~ /\./ # skip directories  
  File.readlines("#{dir}/#{file}").each do |line|  
   next if line =~ /^\s*#/ # skip comments  
   break if count > 10 && DEBUG # testing only (read 10 lines)  
   @fields = CSV.parse(line)[0]  
   warn @fields.inspect if DEBUG # to STDOUT  
   time_str = @fields.slice!(2,2).join(' ')  
   warn time_str if DEBUG  
   begin # NOTE: some of the data has badly formatted dates  
    date = DateTime.strptime(time_str, '%m/%d/%Y %H:%M%P')  
    warn date.strftime '%Y-%m-%d %H:%M' if DEBUG  
    secs = date.to_time.to_i * 1000 # NOTE: must convert seconds to milliseconds  
   rescue  
    warn "#{file} - #{time_str} - #{line}" # prints to STDERR  
    next  
   end  
   begin  
    match = @fields[6].match /([+-]*\d+\.\d+)/  
    change = match[1].to_f # convert change to float  
    warn "change: #{@fields[6]} -> #{match.inspect} -> #{change}" if DEBUG  
   rescue  
    warn "no match for change on: #{line}"  
    change = nil  
   end  
   # NOTE: $date must be in double quotes  
   puts "{ symbol: \"#{@fields[0]}\", name: \"#{@fields[1]}\", price: #{@fields[2].to_f}, dividend_yield: #{@fields[3].to_f}, p_e_ratio: #{@fields[4].to_f}, volume: #{@fields[5].to_i}, day_value_change: #{change}, previous_close: #{@fields[7].to_f}, last_trade_time: { \"$date\": #{secs} } }" # json to stdout  
   count+=1  
  end  
  break if DEBUG # testing only (quit after first file)  
 end  

MONGO AGGREGATION JAVASCRIPT

 // # mongod --nojournal  
 // # mongoimport --db stocks --collection price --type json --file stock_data.json   
 // import looks good  
 > db.price.find()  
 { "_id" : ObjectId("505dded4062e11e40afc5c46"), "symbol" : "AXP", "name" : "American Express ", "price" : 44.73, "dividend_yield" : 2.01, "p_e_ratio" : 14.6, "volume" : 500, "day_value_change" : 0, "previous_close" : 44.73, "last_trade_time" : ISODate("2011-01-06T16:00:00Z") }  
 { "_id" : ObjectId("505dded4062e11e40afc5c48"), "symbol" : "BAC", "name" : "Bank of America C", "price" : 14.44, "dividend_yield" : 0.28, "p_e_ratio" : 0, "volume" : 2942961, "day_value_change" : 0, "previous_close" : 14.44, "last_trade_time" : ISODate("2011-01-06T16:02:00Z") }  
 { "_id" : ObjectId("505dded4062e11e40afc5c49"), "symbol" : "BA", "name" : "Boeing Company (T", "price" : 68.8, "dividend_yield" : 2.44, "p_e_ratio" : 14.9, "volume" : 2810, "day_value_change" : 0, "previous_close" : 68.8, "last_trade_time" : ISODate("2011-01-06T16:01:00Z") }  
 { "_id" : ObjectId("505dded4062e11e40afc5c4a"), "symbol" : "CAT", "name" : "Caterpillar, Inc.", "price" : 93.54, "dividend_yield" : 1.84, "p_e_ratio" : 30.74, "volume" : 2100, "day_value_change" : 0, "previous_close" : 93.54, "last_trade_time" : ISODate("2011-01-06T16:01:00Z") }  
 { "_id" : ObjectId("505dded4062e11e40afc5c4b"), "symbol" : "CVX", "name" : "Chevron Corporati", "price" : 90.69, "dividend_yield" : 3.13, "p_e_ratio" : 10.84, "volume" : 100, "day_value_change" : 0, "previous_close" : 90.69, "last_trade_time" : ISODate("2011-01-06T16:00:00Z") }  
 // list symbols  
 > db.price.aggregate({"$group": {_id: 0, symbols: {"$addToSet": "$symbol"}}});  
 {  
  "result" : [  
   {  
    "_id" : 0,  
    "symbols" : [  
     "WMT",  
     "UTX",  
     "DD",  
     "AXP",  
     "TRV",  
     "PFE",  
     "MRK",  
     "MCD",  
     "KFT",  
     "VZ",  
     "XOM",  
     "JPM",  
     "INTC",  
     "IBM",  
     "AA",  
     "HD",  
     "MSFT",  
     "T",  
     "DIS",  
     "BAC",  
     "CSCO",  
     "HPQ",  
     "MMM",  
     "CVX",  
     "CAT",  
     "KO",  
     "PG",  
     "GE",  
     "JNJ",  
     "BA"  
    ]  
   }  
  ],  
  "ok" : 1  
 }  
 > db.price.ensureIndex({symbol:1})  
 > db.price.aggregate({"$project": {symbol: 1}}, {"$group": {_id: 0, symbols: {"$addToSet": "$symbol"}}});  
 //neither the index nor the project tag sped up this query (more or less the same)  
 > db.price.aggregate({"$project": {symbol: 1}}, {"$group": {_id: "$symbol", counts: {"$sum": 1}}});  
 // list counts per symbol in whole collection  
 > db.price_from_csv.aggregate({"$project": {symbol: 1}}, {"$group": {_id: "$symbol", counts: {"$sum": 1}}});  
 {  
   "result" : [  
     {  
       "_id" : "WMT",  
       "counts" : 263888  
     },  
     {  
       "_id" : "UTX",  
       "counts" : 263888  
     },  
     {  
       "_id" : "DD",  
       "counts" : 263888  
     },  
     {  
       "_id" : "AXP",  
       "counts" : 263888  
     },  
     {  
       "_id" : "TRV",  
       "counts" : 263888  
     },  
     {  
       "_id" : "PFE",  
       "counts" : 263888  
     },  
     {  
       "_id" : "MRK",  
       "counts" : 263888  
     },  
     {  
       "_id" : "MCD",  
       "counts" : 263888  
     },  
     {  
       "_id" : "KFT",  
       "counts" : 263888  
     },  
     {  
       "_id" : "VZ",  
       "counts" : 263888  
     },  
     {  
       "_id" : "XOM",  
       "counts" : 263888  
     },  
     {  
       "_id" : "JPM",  
       "counts" : 263888  
     },  
     {  
       "_id" : "INTC",  
       "counts" : 263888  
     },  
     {  
       "_id" : "IBM",  
       "counts" : 263888  
     },  
     {  
       "_id" : "AA",  
       "counts" : 263888  
     },  
     {  
       "_id" : "HD",  
       "counts" : 263888  
     },  
     {  
       "_id" : "MSFT",  
       "counts" : 263888  
     },  
     {  
       "_id" : "T",  
       "counts" : 263888  
     },  
     {  
       "_id" : "DIS",  
       "counts" : 263888  
     },  
     {  
       "_id" : "BAC",  
       "counts" : 263888  
     },  
     {  
       "_id" : "CSCO",  
       "counts" : 263888  
     },  
     {  
       "_id" : "HPQ",  
       "counts" : 263888  
     },  
     {  
       "_id" : "MMM",  
       "counts" : 263888  
     },  
     {  
       "_id" : "CVX",  
       "counts" : 263888  
     },  
     {  
       "_id" : "CAT",  
       "counts" : 263888  
     },  
     {  
       "_id" : "KO",  
       "counts" : 263888  
     },  
     {  
       "_id" : "PG",  
       "counts" : 263888  
     },  
     {  
       "_id" : "GE",  
       "counts" : 263888  
     },  
     {  
       "_id" : "JNJ",  
       "counts" : 263888  
     },  
     {  
       "_id" : "BA",  
       "counts" : 262690  
     }  
   ],  
   "ok" : 1  
 }  
 // list counts per year for a symbol (obviously some bad dates got into the mix, interesting)  
 > db.price.aggregate({"$match": {symbol: "BA"}}, {"$project": {last_trade_time: 1}}, {"$group": {_id: {"$year": "$last_trade_time"}, count: {"$sum": 1}}});  
 {  
   "result" : [  
     {  
       "_id" : 2012,  
       "count" : 111276  
     },  
     {  
       "_id" : 1970,  
       "count" : 2  
     },  
     {  
       "_id" : 2011,  
       "count" : 151412  
     }  
   ],  
   "ok" : 1  
 }  
 // time for an index  
 > db.price.ensureIndex({last_trade_time:1})  
 // same again for all  
 > db.price.aggregate({"$project": {symbol: 1, last_trade_time: 1}}, {"$group": {_id: {"$year": "$last_trade_time"}, count: {"$sum": 1}}});  
 {  
   "result" : [  
     {  
       "_id" : 2012,  
       "count" : 3338278  
     },  
     {  
       "_id" : 1970,  
       "count" : 48  
     },  
     {  
       "_id" : 2011,  
       "count" : 4577116  
     }  
   ],  
   "ok" : 1  
 }  
 // look for the bad rows (1970)  
 > db.price.find({ last_trade_time: {"$lt":ISODate("2010-01-01T00:00:00.000Z")} })  
 { "_id" : ObjectId("505ddfa6062e11e40a032916"), "symbol" : "MMM", "name" : "ompany Common Sto", "price" : 223658800, "dividend_yield" : 2.56, "p_e_ratio" : 43.21, "volume" : 3498040, "day_value_change" : 90.69, "previous_close" : 90.75, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa6062e11e40a032934"), "symbol" : "MMM", "name" : "ompany Common Sto", "price" : 223658800, "dividend_yield" : 2.56, "p_e_ratio" : 43.21, "volume" : 3498040, "day_value_change" : 90.69, "previous_close" : 90.75, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d7d"), "symbol" : "T", "name" : " Inc.", "price" : 223837152, "dividend_yield" : 21, "p_e_ratio" : 16.86, "volume" : 25700700, "day_value_change" : 28.43, "previous_close" : 28.5, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d7f"), "symbol" : "BA", "name" : "ng Company (The) ", "price" : 223842096, "dividend_yield" : 3.18, "p_e_ratio" : 43.25, "volume" : 5961360, "day_value_change" : 72.46, "previous_close" : 72.66, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d84"), "symbol" : "DIS", "name" : " Disney Company (", "price" : 223834224, "dividend_yield" : 2.12, "p_e_ratio" : 108.7, "volume" : 10490500, "day_value_change" : 43.38, "previous_close" : 43.48, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d85"), "symbol" : "DD", "name" : " du Pont de Nemou", "price" : 223828288, "dividend_yield" : 4.14, "p_e_ratio" : 44.41, "volume" : 6414850, "day_value_change" : 54.52, "previous_close" : 54.62, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d86"), "symbol" : "XOM", "name" : "n Mobil Corporati", "price" : 223838976, "dividend_yield" : 2.54, "p_e_ratio" : 47.39, "volume" : 20538500, "day_value_change" : 82.65, "previous_close" : 83.41, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d88"), "symbol" : "HPQ", "name" : "ett-Packard Compa", "price" : 223828416, "dividend_yield" : 1.35, "p_e_ratio" : 152.53, "volume" : 17942300, "day_value_change" : 48.52, "previous_close" : 48.81, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d89"), "symbol" : "HD", "name" : " Depot, Inc. (The", "price" : 223840032, "dividend_yield" : 6.81, "p_e_ratio" : 39.36, "volume" : 11546200, "day_value_change" : 37.1, "previous_close" : 37.155, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d8b"), "symbol" : "INTC", "name" : "l Corporation", "price" : 223838720, "dividend_yield" : 13.8, "p_e_ratio" : 33.24, "volume" : 54533600, "day_value_change" : 21.73, "previous_close" : 21.77, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d93"), "symbol" : "PG", "name" : "ter & Gamble Comp", "price" : 223838448, "dividend_yield" : 4.6, "p_e_ratio" : 33.65, "volume" : 10235700, "day_value_change" : 64.71, "previous_close" : 64.88, "last_trade_time" : ISODate("1970-01-01T19:01:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d80"), "symbol" : "CAT", "name" : "rpillar, Inc. Com", "price" : 223833680, "dividend_yield" : 1.68, "p_e_ratio" : 59.05, "volume" : 5715380, "day_value_change" : 102.38, "previous_close" : 102.75, "last_trade_time" : ISODate("1970-01-01T19:02:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d82"), "symbol" : "KO", "name" : "-Cola Company (Th", "price" : 223835840, "dividend_yield" : 4.35, "p_e_ratio" : 36.17, "volume" : 9558050, "day_value_change" : 63.47, "previous_close" : 63.66, "last_trade_time" : ISODate("1970-01-01T19:02:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d8c"), "symbol" : "JNJ", "name" : "son & Johnson Com", "price" : 223829632, "dividend_yield" : 5.66, "p_e_ratio" : 28.99, "volume" : 10984400, "day_value_change" : 60.9, "previous_close" : 61.16, "last_trade_time" : ISODate("1970-01-01T19:02:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033da0"), "symbol" : "KO", "name" : "-Cola Company (Th", "price" : 223835840, "dividend_yield" : 4.35, "p_e_ratio" : 36.17, "volume" : 9558050, "day_value_change" : 63.47, "previous_close" : 63.66, "last_trade_time" : ISODate("1970-01-01T19:02:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d95"), "symbol" : "UTX", "name" : "ed Technologies C", "price" : 223840112, "dividend_yield" : 2.36, "p_e_ratio" : 50.08, "volume" : 3661150, "day_value_change" : 85.1, "previous_close" : 85.13, "last_trade_time" : ISODate("1970-01-01T19:03:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d81"), "symbol" : "CVX", "name" : "ron Corporation C", "price" : 223829360, "dividend_yield" : 3.02, "p_e_ratio" : 34.21, "volume" : 7859110, "day_value_change" : 96.4, "previous_close" : 97.17, "last_trade_time" : ISODate("1970-01-01T19:04:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d7b"), "symbol" : "AA", "name" : "a Inc. Common Sto", "price" : 223827792, "dividend_yield" : 4.03, "p_e_ratio" : 144.92, "volume" : 28247700, "day_value_change" : 17.22, "previous_close" : 17.39, "last_trade_time" : ISODate("1970-01-01T19:06:00Z") }  
 { "_id" : ObjectId("505ddfa7062e11e40a033d90"), "symbol" : "MRK", "name" : "k & Company, Inc.", "price" : 223836448, "dividend_yield" : 13.91, "p_e_ratio" : 21.77, "volume" : 18065100, "day_value_change" : 33.02, "previous_close" : 33.085, "last_trade_time" : ISODate("1970-01-01T19:06:00Z") }  
 { "_id" : ObjectId("505de199062e11e40a210d32"), "symbol" : "MMM", "name" : "ompany Common Sto", "price" : 505544928, "dividend_yield" : 2.56, "p_e_ratio" : 42.99, "volume" : 3294730, "day_value_change" : 92.32, "previous_close" : 92.43, "last_trade_time" : ISODate("1970-01-01T20:01:00Z") }  
 > db.price.remove({ last_trade_time: {"$lt":ISODate("2010-01-01T00:00:00.000Z")} })  
 // multi-key group and sort  
 > db.price.aggregate({"$project": {symbol: 1, last_trade_time: 1}}, {"$group": {_id: {symbol: "$symbol", year: {"$year": "$last_trade_time"}}, count: {"$sum":1}}}, {"$sort": {_id: 1}});  
 {  
   "result" : [  
     {  
       "_id" : {  
         "symbol" : "AA",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "AA",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "AXP",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "AXP",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "BA",  
         "year" : 2011  
       },  
       "count" : 151412  
     },  
     {  
       "_id" : {  
         "symbol" : "BA",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "BAC",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "BAC",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "CAT",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "CAT",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "CSCO",  
         "year" : 2011  
       },  
       "count" : 152612  
     },  
     {  
       "_id" : {  
         "symbol" : "CSCO",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "CVX",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "CVX",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "DD",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "DD",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "DIS",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "DIS",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "GE",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "GE",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "HD",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "HD",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "HPQ",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "HPQ",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "IBM",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "IBM",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "INTC",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "INTC",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "JNJ",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "JNJ",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "JPM",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "JPM",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "KFT",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "KFT",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "KO",  
         "year" : 2011  
       },  
       "count" : 152609  
     },  
     {  
       "_id" : {  
         "symbol" : "KO",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "MCD",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "MCD",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "MMM",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "MMM",  
         "year" : 2012  
       },  
       "count" : 111275  
     },  
     {  
       "_id" : {  
         "symbol" : "MRK",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "MRK",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "MSFT",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "MSFT",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "PFE",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "PFE",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "PG",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "PG",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "T",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "T",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "TRV",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "TRV",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "UTX",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "UTX",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "VZ",  
         "year" : 2011  
       },  
       "count" : 152612  
     },  
     {  
       "_id" : {  
         "symbol" : "VZ",  
         "year" : 2012  
       },  
       "count" : 111275  
     },  
     {  
       "_id" : {  
         "symbol" : "WMT",  
         "year" : 2011  
       },  
       "count" : 152611  
     },  
     {  
       "_id" : {  
         "symbol" : "WMT",  
         "year" : 2012  
       },  
       "count" : 111276  
     },  
     {  
       "_id" : {  
         "symbol" : "XOM",  
         "year" : 2011  
       },  
       "count" : 152610  
     },  
     {  
       "_id" : {  
         "symbol" : "XOM",  
         "year" : 2012  
       },  
       "count" : 111276  
     }  
   ],  
   "ok" : 1  
 }  
 // using match (2012 only)  
 > db.price.aggregate({"$match": {last_trade_time: {"$gte":ISODate("2012-01-01T00:00:00.000Z")}}}, {"$group": {_id: "$symbol", count: {"$sum": 1}}});  
 {  
   "result" : [  
     {  
       "_id" : "MMM",  
       "count" : 111275  
     },  
     {  
       "_id" : "UTX",  
       "count" : 111276  
     },  
     {  
       "_id" : "DD",  
       "count" : 111276  
     },  
     {  
       "_id" : "AXP",  
       "count" : 111276  
     },  
     {  
       "_id" : "TRV",  
       "count" : 111276  
     },  
     {  
       "_id" : "PFE",  
       "count" : 111276  
     },  
     {  
       "_id" : "MRK",  
       "count" : 111276  
     },  
     {  
       "_id" : "MCD",  
       "count" : 111276  
     },  
     {  
       "_id" : "KFT",  
       "count" : 111276  
     },  
     {  
       "_id" : "VZ",  
       "count" : 111275  
     },  
     {  
       "_id" : "XOM",  
       "count" : 111276  
     },  
     {  
       "_id" : "JPM",  
       "count" : 111276  
     },  
     {  
       "_id" : "INTC",  
       "count" : 111276  
     },  
     {  
       "_id" : "IBM",  
       "count" : 111276  
     },  
     {  
       "_id" : "AA",  
       "count" : 111276  
     },  
     {  
       "_id" : "HD",  
       "count" : 111276  
     },  
     {  
       "_id" : "MSFT",  
       "count" : 111276  
     },  
     {  
       "_id" : "BAC",  
       "count" : 111276  
     },  
     {  
       "_id" : "CSCO",  
       "count" : 111276  
     },  
     {  
       "_id" : "T",  
       "count" : 111276  
     },  
     {  
       "_id" : "DIS",  
       "count" : 111276  
     },  
     {  
       "_id" : "HPQ",  
       "count" : 111276  
     },  
     {  
       "_id" : "WMT",  
       "count" : 111276  
     },  
     {  
       "_id" : "PG",  
       "count" : 111276  
     },  
     {  
       "_id" : "GE",  
       "count" : 111276  
     },  
     {  
       "_id" : "JNJ",  
       "count" : 111276  
     },  
     {  
       "_id" : "CVX",  
       "count" : 111276  
     },  
     {  
       "_id" : "CAT",  
       "count" : 111276  
     },  
     {  
       "_id" : "KO",  
       "count" : 111276  
     },  
     {  
       "_id" : "BA",  
       "count" : 111276  
     }  
   ],  
   "ok" : 1  
 }  
 // here's one that's actually a bit useful: Bank of America high/low/average price by week in 2012  
 > db.price.aggregate({"$match": {last_trade_time: {"$gte":ISODate("2012-01-01T00:00:00.000Z")}, symbol: "BA"}}, {"$group": {_id: {"$week":"$last_trade_time"}, high: {"$max": "$price"}, low: {"$min": "$price"}, avg: {"$avg": "$price"}}}, {"$sort":{_id:1}});  
 {  
   "result" : [  
     {  
       "_id" : 1,  
       "high" : 74.98,  
       "low" : 72.79,  
       "avg" : 73.99888326980202  
     },  
     {  
       "_id" : 2,  
       "high" : 75.68,  
       "low" : 74.17,  
       "avg" : 74.78443128234832  
     },  
     {  
       "_id" : 3,  
       "high" : 75.981,  
       "low" : 74.83,  
       "avg" : 75.4167605833319  
     },  
     {  
       "_id" : 4,  
       "high" : 76.312,  
       "low" : 72.96,  
       "avg" : 75.23588637120919  
     },  
     {  
       "_id" : 5,  
       "high" : 76.73,  
       "low" : 73.69,  
       "avg" : 75.07820737403995  
     },  
     {  
       "_id" : 6,  
       "high" : 75.9,  
       "low" : 74.325,  
       "avg" : 75.17866743023808  
     },  
     {  
       "_id" : 7,  
       "high" : 75.96,  
       "low" : 74.78,  
       "avg" : 75.24222214802415  
     },  
     {  
       "_id" : 8,  
       "high" : 76.636,  
       "low" : 75.09,  
       "avg" : 75.92514222592771  
     },  
     {  
       "_id" : 9,  
       "high" : 75.76,  
       "low" : 74.661,  
       "avg" : 75.09111806064543  
     },  
     {  
       "_id" : 10,  
       "high" : 74.9501,  
       "low" : 72.3,  
       "avg" : 73.6226790136067  
     },  
     {  
       "_id" : 11,  
       "high" : 75.79,  
       "low" : 73.2,  
       "avg" : 74.7180465488498  
     },  
     {  
       "_id" : 12,  
       "high" : 75.68,  
       "low" : 73.39,  
       "avg" : 74.69681463821372  
     },  
     {  
       "_id" : 13,  
       "high" : 75.27,  
       "low" : 73,  
       "avg" : 74.48685070187024  
     },  
     {  
       "_id" : 14,  
       "high" : 75.4645,  
       "low" : 73.3,  
       "avg" : 74.13860771929662  
     },  
     {  
       "_id" : 15,  
       "high" : 73.77,  
       "low" : 70.6,  
       "avg" : 72.4210965729701  
     },  
     {  
       "_id" : 16,  
       "high" : 74.36,  
       "low" : 72.325,  
       "avg" : 73.49481252923461  
     },  
     {  
       "_id" : 17,  
       "high" : 77.54,  
       "low" : 72.17,  
       "avg" : 75.31753740765522  
     },  
     {  
       "_id" : 18,  
       "high" : 77.82,  
       "low" : 75.52,  
       "avg" : 76.7284854496451  
     },  
     {  
       "_id" : 19,  
       "high" : 76.04,  
       "low" : 73.28,  
       "avg" : 74.55083031416956  
     },  
     {  
       "_id" : 20,  
       "high" : 73.76,  
       "low" : 68.94,  
       "avg" : 71.73182498332316  
     },  
     {  
       "_id" : 21,  
       "high" : 72.18,  
       "low" : 69.67,  
       "avg" : 70.88400550458809  
     },  
     {  
       "_id" : 22,  
       "high" : 70.4825,  
       "low" : 67.1999,  
       "avg" : 69.18734449999921  
     },  
     {  
       "_id" : 23,  
       "high" : 70.3,  
       "low" : 66.84,  
       "avg" : 68.64987898599199  
     },  
     {  
       "_id" : 24,  
       "high" : 72.79,  
       "low" : 69.84,  
       "avg" : 71.68434567901214  
     },  
     {  
       "_id" : 25,  
       "high" : 73.59,  
       "low" : 71.1,  
       "avg" : 72.26916556666535  
     },  
     {  
       "_id" : 26,  
       "high" : 74.36,  
       "low" : 70.5,  
       "avg" : 71.82509259752993  
     },  
     {  
       "_id" : 27,  
       "high" : 74.74,  
       "low" : 72.5,  
       "avg" : 73.90557605868787  
     },  
     {  
       "_id" : 28,  
       "high" : 75.02,  
       "low" : 70.92,  
       "avg" : 72.89437934022011  
     },  
     {  
       "_id" : 29,  
       "high" : 75.07,  
       "low" : 72.0631,  
       "avg" : 73.68273189856532  
     },  
     {  
       "_id" : 30,  
       "high" : 75.93,  
       "low" : 71.37,  
       "avg" : 73.76041403333438  
     },  
     {  
       "_id" : 31,  
       "high" : 75.55,  
       "low" : 71.23,  
       "avg" : 73.37943595468138  
     },  
     {  
       "_id" : 32,  
       "high" : 74.69,  
       "low" : 72.72,  
       "avg" : 74.03845807204802  
     },  
     {  
       "_id" : 33,  
       "high" : 74.43,  
       "low" : 72.76,  
       "avg" : 73.7013437333349  
     },  
     {  
       "_id" : 34,  
       "high" : 74.325,  
       "low" : 70.04,  
       "avg" : 72.34569156666593  
     },  
     {  
       "_id" : 35,  
       "high" : 71.95,  
       "low" : 70.53,  
       "avg" : 71.3327890464269  
     },  
     {  
       "_id" : 36,  
       "high" : 73.25,  
       "low" : 70.4199,  
       "avg" : 71.99506040016878  
     },  
     {  
       "_id" : 37,  
       "high" : 72.39,  
       "low" : 70.4651,  
       "avg" : 71.27986476666649  
     },  
     {  
       "_id" : 38,  
       "high" : 70.7,  
       "low" : 69.04,  
       "avg" : 69.96584609175824  
     }  
   ],  
   "ok" : 1  
 }  
 // use sorting to find extremes  
 // high by week  
 > db.price.aggregate({"$match": {last_trade_time: {"$gte":ISODate("2012-01-01T00:00:00.000Z")}, symbol: "BA"}}, {"$group": {_id: {"$week":"$last_trade_time"}, high: {"$max": "$price"}, low: {"$min": "$price"}, avg: {"$avg": "$price"}}}, {"$sort":{high:-1}}, {"$group": {_id: 0, high: {"$first":"$high"}, week: {"$first": "$_id"}}});  
 {  
   "result" : [  
     {  
       "_id" : 0,  
       "high" : 77.82,  
       "week" : 18  
     }  
   ],  
   "ok" : 1  
 }  
 // low by week  
 > db.price.aggregate({"$match": {last_trade_time: {"$gte":ISODate("2012-01-01T00:00:00.000Z")}, symbol: "BA"}}, {"$group": {_id: {"$week":"$last_trade_time"}, high: {"$max": "$price"}, low: {"$min": "$price"}, avg: {"$avg": "$price"}}}, {"$sort":{low:1}}, {"$group": {_id: 0, low: {"$first":"$low"}, week: {"$first": "$_id"}}});  
 {  
   "result" : [  
     {  
       "_id" : 0,  
       "low" : 66.84,  
       "week" : 23  
     }  
   ],  
   "ok" : 1  
 }  

CONCLUSION

Each element in the aggregation pipeline is a further transformation of the data. At each stage's output, documents are passed into the next stage's input.

Obviously, there are probably many more interesting queries one could concoct, but hopefully this gives a reasonable overview of the concept of using the new Aggregation Framework.

I hope this is helpful to anyone looking at playing around with Mongo in general, or the Aggregation Framework in particular.

The State of Full-Stack JavaScript

[tl;dr: A brief look at JavaScript-based frameworks.]

I was having a conversation with a guy at work the other day, and I suggested (only somewhat jokingly) that the future of web development would be JavaScript-based, not just client-side, but server-side and database, too. I kind of surprised myself by being somewhat excited about the prospect.

I, like most developers I know, learned to love JavaScript (again?) through jQuery http://jquery.com/. jQuery made writing client-side code fun, similar to the way that Ruby On Rails http://rubyonrails.org/ made writing server-side code fun.

We recently started using MongoDB at work for a specific project http://www.mongodb.org/. We’re calling the project the “Analytics Engine.” It was originally called “Ad-hoc Reporting.” It’s actually very similar to a project I did years ago while working in the Academic Computing department at Dartmouth College. That project was for an undergrad course called “War and Peace in the 20th Century,” and it was meant to introduce students to basic data analysis concepts by allowing them to choose a couple of variables from a dataset and get some statistical information about the individual variables as well as the relationship between them.

You can have a look here: http://dataanalysis.q-media.com/

It’s using a database of terrorist incidents which had come from the Rand Corporation (before Rand had set up their own site for doing this type of analysis). http://www.rand.org/nsrd/projects/terrorism-incidents.html

Please bear in mind that this was just a prototype, and it was also my first ever Rails app (Spring 2006). It’s primative. The full-blown application never got built because the professor who was sponsoring it moved to Stanford shortly thereafter. I’m honestly surprised it still works at all. It’s got a sqlite backend, and it’s based on the idea that all the variables are either categorical or numeric (we left out time-series data, except for filters, as “Year” was entered as a categorical variable).

Anyway, the upshot of all that is that it turns out that databases can be fun, too! MongoDB has proved to be a really slick way to implement this kind of data analysis application. Because the data that we’re doing reporting on comes from a forms-based input system (built on Oracle APEX, don’t ask!), we had to be able to easily handle changes in the schema. Presto! Just don’t use a schema!

So, back to JavaScript (which is, after all, the point of this post)…

Some time ago, not long after that Terrorism application was built, people started to contemplate something outrageous: that JavaScript could (and should) be taken seriously.

We had Steve Yegge talking about the “NBL” in February, 2007: http://steve-yegge.blogspot.com/2007/02/next-big-language.html

Who knew he was talking about JavaScript at the time? C’mon, be honest!

We had JavaScript: The Good Parts by Douglas Crockford (© 2008): http://www.amazon.com/JavaScript-Good-Parts-Douglas-Crockford/dp/0596517742

(see also this Google Tech Talk: http://www.youtube.com/watch?v=hQVTIJBZook)

Now, we even have the (shock, horror) idea that MongoDB might begin to be taken seriously: http://www.theregister.co.uk/2012/06/11/mongo_db/

Other developments worth mentioning: CommonJS promises a JavaScript standard library[http://www.commonjs.org/], Rhino [https://developer.mozilla.org/en-US/docs/Rhino], which has been around for a while, provides interesting possibilities for Java integration, and, of course, Node.js [http://nodejs.org/], which is built on top of Google’s blazingly fast V8 JS engine, is the cat’s pajamas (according to many: http://notinventedhe.re/on/2011-7-26) for programming network services.

See the benchmarks on V8 JS here: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&lang2=v8

So, what does it really take to put together a full-stack of piping hot JavaScript pancakes?

Well, first of all, to be fair, I’m slightly misusing the expression “full-stack.” Full-stack can mean Model (ORM), View (templates), and Controller, and Rails qualifies as “full-stack” by some definitions. I’m using a broader interpretation to mean not just the server-side framework, but the database interface language (where SQL served in the past), and, of course, client-side code (where JavaScript is the de facto standard).

So, we’re looking for the full banana. And, when I ask the question, “What is the state of full-stack JavaScript development today?” (to my friend, Google), what I get back, is this (more than two years old): http://jimbojw.com/fullstack/ which gives a nice overview of the issue.

The central idea is that it is now possible to use HTML, CSS, and JavaScript to build entire web applications, including interacting with the datastore.

OK. We get it. What about specifics?

A little more Googling, leads me, as with most questions about programming, to stackoverflow...

http://stackoverflow.com/questions/4180682/are-there-any-full-stack-server-side-javascript-frameworks-available

That’s very helpful, but it’s interesting to note that that post is from almost two years ago.

To be frank, I wasn’t blown away by the response I got. There didn’t seem to be a “clear winner” in the race to become the default solution. The playing field seems a few players short of a full squad.

Nonetheless, a few names did keep cropping up.

I shall endeavor to list the major players here (please let me know in the comments if there are any other things I should mention).

Derby: http://derbyjs.com/

“Derby is built on top of popular libraries, including Node.js, Express, Socket.IO, Browserify, Stylus, LESS,UglifyJS, MongoDB, and soon other popular databases and datastores. These libraries can also be used directly. The data synchronization layer, Racer, can be used separately. Other client libraries, such as jQuery, and other Node.js modules from npm work just as well along with Derby.”

Derby’s docs also includes a handy section called “Why Not Use Rails and Backbone?” that supports the central thesis of this post.

Tower.js: http://towerjs.org/

“Default Development Stack

• MongoDB (database)
• Redis (background jobs)
• CoffeeScript
• Stylus
• Jasmine (tests)
• jQuery“

Ringo (Helma): http://ringojs.org/ http://www.helma.org/

I’m struggling a bit to figure out exactly how these projects are related to one another (Ringo appears to be a newer implementation of Helma, but I’m not quite sure if it’s meant to supercede it). In any case, Ringo and Helma seem to be the big players based on Rhino (therefore Java) and use JDBC for database connectivity, as well as Jetty.

So, my mission, should I decide to accept it, is to attempt to evaluate this stuff over the next few weeks and post anything interesting that I find. In particular, which, if any, of these solutions seems Ready for Prime Time.

If I can bring myself to do it, I may re-code the above referenced Terrorism database application using one of these stacks. Hopefully, it will be more fun than the first time!

Fun with Financial Data (or As Much Fun As You Can Have Without Actually Making Any Money)

I've started reading the book Data Analysis with Open Source Tools by Philipp K. Janert. It's very well written and informative, and I highly recommend it. I've only scratched the surface so far, but I decided that I wanted to try to apply some ideas from it to a problem that had been kicking around in my head for while.

And then I figured: why not write a blog post about it! (After all, no one reads this thing anyway ;-)

This problem occurred to me a while back while reading another O'Reilly book: Programming Collective Intelligence by Toby Segaran. The idea involves looking at a couple of pieces of information that are available on the web: Financial price data and financial news (both of which are freely available at Yahoo! Finance). I realize that the data that Yahoo publishes is not particularly real-time and one wouldn't probably want to build an HFT system on it, but it will serve to do a little studying.

I didn't really have a hypothesis in mind when I started. Taking a cue from Mr. Janert, I decided to take a look at the data with an open mind (no pre-conceived notions). At the heart of this inquiry, however, is a two-tier question:

1. Is there some kind of correlation between when news gets published online and when stock prices move? and
2. Is there a predictable sequential relationship that can be construed?

Why do this? Well, it's interesting. I'd been interested for a while in finding a worthwhile excuse to do something with financial data. Besides, there's a small chance that it might reveal an interesting opportunity (if news precedes predictable changes in price? Cha-ching!). More likely, however, I was expecting that I'd find that We, The Sheeple are trailing the HFT systems and insiders and there is little or no hope for us little folks to make a thin dime playing the market.

Please note that this is a first pass at this. By no means am I imagining that this is a comprehensive approach. Indeed, I would be very grateful if anyone has any ideas for what to look at next (please comment!).

For those who like the punchline first, here's the spoiler: the evidence is inconclusive as to whether any predictions can be made in price change based on headlines. Further study might reveal something, but at first glance, it just doesn't add up (pun intended).

Conversely, and somewhat surprisingly, it is not conclusive that price changes precede news. In other words, it is not clear cut that the news follows the market.

However, there is clearly a correlation in the data: big price shifts, high volume, and an increase in published news tend to go hand-in-hand. This is an unremarkable finding, but nonetheless, it was an interesting exercise.

Now, despite having warned you that these findings are dull as dishwater, if you are really interested, I presume you are still with me...

To analyze the problem, the first thing I did was to set up a method for capturing data. Once I had a bit of data to look at, then I could begin my inquiry.

I don't really want to spend a lot of time discussing the specific technologies employed, as that's not really the point of this post, but I will just summarize this step briefly:

1. A couple of shell scripts employing curl (via cron) to grab data at reasonable intervals: every minute for price data, and every hour for news data.
2. A couple of Scala scripts for parsing that data and dumping into a sqlite database.
3. SQL queries to grab the data (grouped as appropriate) in CSV format to pull into Excel (where I can create charts, where I can create charts for visualization, etc.)

As you can see, very low-tech, indeed. All in the spirit of keeping it simple. I thought about using matplotlib, which is a great Python-based tool for doing data analysis that I've had the pleasure of using a number of times in the past. But, as the data set I was focusing on was pretty small, Excel did the job nicely.

In another iteration of this inquiry, I might be inclined to try out R or Octave, and dig a little deeper. Perhaps I'll even take a further look at what data is available and try to expand the reach. If I'm feeling especially crazy, I might even give Hadoop a try.

For purposes of this exercise, I will be looking at this data over the course of a single trading week: January 10-14 2011 (inclusive), and I'm just grabbing data on stocks that make up the Dow Jones Industrial Average. No particular reason other than that that was a convenient week to do it, and it seemed like a sufficient amount of time for this initial undertaking.

I figured I would take a "big picture" view first. See whether just glancing at the data indicates anything interesting. If something jumps out, then drill down further and see where it takes me.

To put this in context, here's a graph showing the Dow's movement around that time:

Prices had been coming down the previous week, but were poised to generally go up this week (hindsight being 20/20 and all).

The first things I want to look at are:

1. How many news headlines are there per stock per day? (And look for spikes in these numbers.)
2. Significant shifts in prices.
3. Significant shifts in volumes.

That alone should establish where there's a relationship that's worth exploring. Doing this on a per stock per day over these few days should be sufficient to start. Then, if motivated, look at the intra-day prices and specific publish times of articles to see if one can determine whether it's the chicken or the egg that comes first.

To start, here's some summary information about our headline data per stock per day (biggest ranges, and largest gaps between mean and max are shown highlighted):

And here it is again, measuring number of standard deviations from the mean (largest numbers highlighted):

Those spikes are a lot clearer when looking at them charted:

Looking at these, just based on spikes in number of news articles, I'd be interested in looking further at the following:

(Because I know what headlines at Yahoo tend to get grouped with stocks in the same industries, I'm looking for those relationships as well.)

1. DuPont (DD), Johnson & Johnson (JNJ), and Procter & Gamble (PG) on Monday.
2. AT&T (T) and Verizon (VZ) on Tuesday.
3. Merck (MRK) and Pfizer (PFE) on Thursday.
4. Bank of America (BAC) and JP Morgan (JPM) on Friday.

Now, let's see if that jives with what we can see in price data (note: this is shown by price range for the day as a fluctuation (max-min) percentage from the previous day's close):

Looking at the price data, there is certainly enough coming through to believe that there is a correlation here. Much of the data points to the same stocks/days. I might be inclined to filter my list, however, as it least some of the headline data is likely to be associated with certain stocks only because of the industry relationship:

1. DuPont (DD) on Monday.
2. AT&T (T) and Verizon (VZ) on Tuesday.
3. Merck (MRK) on Thursday.
4. Bank of America (BAC) and JP Morgan (JPM) on Friday.

That reduces it to six different stocks on four different days, which seems like a reasonable number to analyze further (with two industry relationships to look at).

I might also be inclined to add some based on the price data alone, like McDonald's (MCD) or American Express (AXP) on Friday, but when looking at the headline data, there's not really enough to go on. (Most of the headlines for McDonald's on the Friday were actually about Starbucks.)

And, just to complete the picture, I'll have a look at volumes as well.

Again, a lot of the same players emerge, which leads me to think that I'm on the right track. The correlation is clear. Now, I'll have to look at the intra-day data to see if one can draw any conclusions moment-to-moment on the specific scenarios that I've identified.

DuPont (DD): Monday January 10th, 2011

First off, the data... (Note that the price column is price range during that hour as a percentage)

 

The following chart shows the deltas in headline count, price, and volume as a percentage of the total change over the course of the day (this allows unlike numbers to be compared in like manner). In particular, note that I've summed the delta of pre-open headlines into the first hour's number, which might exaggerate it's effect on the chart. But looking at the data you can certainly see that there was significant activity pre-open.

And, price for the day...

One thing's for certain, there was definitely a spike in the number of headlines at 11 am which followed the steep change in price. But most of the headlines from that day were before the bell, and the big dip in price happened before 10 am. By 11 am the price was fairly stable.

So, what were the headlines? Well, here are all the headlines that were published before 11 am:

• 07:07 "Duke, Progress Energy, Sara Lee, Strayer, WellPoint: U.S. Equity Preview"
• 07:15 "US STOCKS-Futures lower on Portugal concerns despite M&A"
• 07:24 "CNNMoney Pre-Market Report - Jan. 10, 2011"
• 07:25 "DuPont to Buy Danisco for $5.8 Billion"
• 07:25 "Futures Fall; DuPont, Earnings in Focus"
• 07:32 "Disappointing Jobs Data Weighs on Futures"
• 07:45 "[$$] Danisco to Recommend DuPont Bid"
• 07:56 "US STOCKS-Futures lower as Portugal concerns offset M&A"
• 08:08 "Let's Make a Deal"
• 08:19 "Futures follow Asia, Europe lower"
• 08:27 "UPDATE 1-Goldman Sachs raises US chemicals sector to attractive"
• 08:30 "[$$] The Day Ahead: Merger Monday"
• 08:40 "UPDATE 3-DuPont's Danisco bid sends sector shares higher"
• 08:53 "Before the Bell: Duke Energy, DuPont, Verizon in spotlight"
• 08:59 "U.S. Stocks Poised For Weak Open"
• 09:09 "[video] News Hub: Before the Opening Bell [1.4 min]"
• 09:14 "[video] Faber Report"
• 09:20 "Danish currency firms on Danisco takeover"
• 09:40 "CNNMoney Market Report - Jan. 10, 2011"
• 09:44 "Dow Falls at Open Despite Rise in M&A"
• 09:47 "Verizon iPhone Announcement Seen for Tuesday"
• 09:48 "Stocks Dip on Euro Debt Concerns"
• 09:50 "Portugal Bailout Push Mars Merger Monday"
• 09:59 "[audio] Opening Bell from MarketWatch Radio Network [1.0 min]"
• 10:00 "Monday Morning Quarterback: Mergers, Foreclosure and Hank Moody"
• 10:16 "Wall St down as Portugal woes offset M&A deals"
• 10:39 "US equities fall on eurozone fears"
• 10:57 "Stocks to Watch: Union Pacific, Exxon and More ..."

Clearly the buyout of Danisco was a big factor here, but I'm having a hard time understanding why it had the effect it did on price. It almost seems counter-intuitive that the share price should fall so dramatically in the first 20 minutes of trading, given the seemingly "good news" of this buyout. Perhaps it's something to do with exchange rates and the European problems that were weighing on the market? Was it this: Danish currency firms on Danisco takeover? (09:20)

Interesting, but by no means conclusive.

One headline in there jumps out, however, and makes a nice segue:

09:47 "Verizon iPhone Announcement Seen for Tuesday"

That'll explain our next subject: Verizon and AT&T activity on Tuesday.

 Verizon (VZ): Tuesday January 11th, 2011

 The spike in price at around 11 am when all those articles were published (which corresponded to the time of the Verizon iPhone launch) is certainly interesting. But I don't think it would take a genius to figure out that that might happen.

The other scenarios below also confirm what was stated previously, so I'm happy to leave the rest of this as an exercise for the reader. There is a correlation between headline count and price changes, but not a clearly sequential one. Again, I welcome feedback or ideas. I hope to come back to this at some stage and delve a little deeper. In the meantime, here's the rest of that data, starting with the Verizon headlines...

• 00:01 "IPhone May Cost Verizon Wireless $5 Billion in Subsidies in First Year"
• 00:15 "ANALYSIS - AT&T faces tough year after losing iPhone exclusive"
• 02:47 "Telecoms' Dividend Yields"
• 03:03 "With iPhone Taunts, AT&T-Verizon Rivalry Escalates"
• 03:51 "11 High Yield Stocks to Consider Now"
• 05:00 "[video] Verizon iPhone Data Plan"
• 06:00 "Nielsen: Young People Across The Globe Love Their Cell Phones (But Use Them Differently)"
• 06:45 "Half of Dow Companies to Post Profit Gains"
• 06:50 "Ahead of the Bell: Verizon iPhone"
• 07:27 "10 Things You Need To Know Before The Opening Bell"
• 07:27 "10 Things You Need To Know Before The Opening Bell"
• 07:30 "CNNMoney Pre-Market Report - Jan. 11, 2011"
• 07:32 "Lowenstein Says Verizon's Network Ready to Handle IPhone: Video"
• 07:39 "10 Things You Need To Know This Morning"
• 07:40 "iPhone Raises Stakes of AT&T-Verizon Rivalry"
• 07:45 "Verizon big winner from having iPhone? Not so fast"
• 07:58 "AT&T and Verizon Trade Taunts Over iPhone"
• 08:03 "[video] 3 Stocks I Saw on TV"
• 08:22 "Will Verizon Demolish AT&T? It Did So Long Ago"
• 08:24 "The Big Questions About Verizon's IPhone"
• 08:24 "Verizon big winner from having iPhone? Not so fast"
• 08:25 "McCourt Says Verizon's Network Can Handle Apple's IPhone: Video"
• 08:36 "How Important Is Offering the iPhone on Verizon to Apple?"
• 08:45 "[$$] ETF Play of the Day: XLK"
• 08:56 "Liveblog: Verizon set to launch the iPhone. Finally."
• 09:02 "[video] AM Report: Chinese Stealth Jet Takes Flight [8.0 min]"
• 09:17 "[video] 3 Networking Stocks Set for Cloud Boost"
• 09:29 "Verizon iPhone Faces Even Bigger Foe"
• 09:33 "Trujillo Says 'Many Questions' Remain For Verizon IPhone: Video"
• 09:45 "Verizon IPhone Event Live Blog"
• 09:45 "[video] News Hub: Verizon iPhone: How will AT&T Respond? [2.7 min]"
• 09:52 "Verizon's Big iPhone Unveil"
• 09:52 "[video] Verizon's iPhone Buzz"
• 09:54 "Mobile Broadband Subs Seen Hitting 1 Billion In 2011"
• 09:58 "[audio] Opening Bell from MarketWatch Radio Network [1.0 min]"
• 10:03 "Video: Is the Verizon iPhone a Threat to Android?"
• 10:05 "Verizon iPhone expected to have unlimited data plan"
• 10:14 "[video] iPhone or Not, Verizon Is A Winner"
• 10:19 "Verizon iPhone: Live Blog"
• 10:37 "Verizon: Stay Tuned"
• 10:47 "LIVE: The Verizon iPhone Event"
• 10:53 "Faber Report: What Will Drive Verizon?"
• 10:54 "LIVE FROM NEW YORK: Verizon Gets the iPhone"
• 10:54 "Verizon iPhone: Will It Fix Verizon's $100 Billion Conundrum?"
• 10:55 "Live Blog: Verizon's iPhone Announcement"
• 10:57 "Live Blogging the Verizon iPhone Announcement"
• 11:18 "Verizon to launch iPhone 4 in February"
• 11:20 "Verizon iPhone: The basics"
• 11:24 "The Biggest Surprise About the Verizon iPhone: It's a Mobile Hotspot"
• 11:24 "Verizon to Carry Apple iPhone, Ending Months of Anticipation"
• 11:30 "Verizon Will Be Selling The iPhone Starting February 10th"
• 11:36 "Live Blog: Verizon's iPhone Announcement"
• 11:38 "UPDATE 2-Verizon Wireless to sell iPhone in February"
• 11:39 "PHOTOS: Verizon iPhone Has Different (Better?) Antenna Design Than AT&T iPhone"
• 11:39 "Verizon, AT&T, And Apple: Guess Which One's Stock Isn't Tanking Today"
• 11:41 "Verizon to Sell iPhone 4 on Feb. 10"
• 11:43 "Verizon Wireless to Offer iPhone 4 in February"
• 11:45 "Verizon iPhone: 'It Begins'"
• 11:50 "Verizon to begin selling iPhone 4 in February"
• 11:51 "Bizarre Fact of the Day: Tyler Durden and Louis Winthorpe III Are Practically the Same Person"
• 11:51 "Verizon to sell Apple iPhone"
• 11:51 "Verizon: Began Discussions With Apple In 2008; Tim Cook Talks"
• 11:54 "Verizon Wireless to sell iPhone in February"
• 12:17 "AT&T Preps for Verizon iPhone With New Ads, Price Cuts"
• 12:18 "[audio] Enderle: Verizon and Apple no match made in heaven [5.1 min]"
• 12:19 "Apple's Phil Schiller on Building the Verizon iPhone"
• 12:20 "AT&T Shrugs Off Verizon iPhone"
• 12:20 "COMPARISON: Verizon Vs. AT&T iPhone"
• 12:26 "FACTBOX - AT&T and Verizon wage battle over iPhone"
• 12:28 "Verizon iPhone: Yep, The Real Target is Google."
• 12:28 "Why No 4G iPhone?"
• 12:30 "AT&T: Five Reasons the Sky Won't Fall When iPhone Goes to Verizon"
• 12:35 "[audio] Midday Update from MarketWatch Radio Network [1.0 min]"
• 12:37 "FACTBOX - AT&T and Verizon wage battle over iPhone"
• 12:37 "Verizon Wireless to sell iPhone in February"
• 12:43 "Verizon to start selling iPhone on Feb. 10"
• 12:48 "Corrected: Factbox: AT&T and Verizon wage battle over iPhone"
• 12:50 "Verizon Will Offer the iPhone Next Month"
• 12:57 "Corrections: Verizon to begin selling iPhone 4 in February"
• 12:59 "Verizon Wireless to sell iPhone in February"
• 13:17 "Stronger earnings reports push stocks higher"
• 13:18 "Most active New York Stock Exchange-traded stocks"
• 13:18 "[video] Apple's iPhone 4 Comes to Verizon [1.4 min]"
• 13:19 "iPhone Will Bring 'Value' to Verizon Shareholders: CEO"
• 13:22 "Apple/Verizon Pt. 2: Should I stay or should I go now?"
• 13:23 "FACTBOX: AT&T and Verizon wage battle over iPhone"
• 13:23 "Halftime: Verizon iPhone To Drag Down Verizon Stock 10%?"
• 13:25 "[$$] iPhone Disappears Over the Verizon"
• 13:28 "[audio] Time for a tech detox? [1.9 min]"
• 13:29 "Wireless Struggles Against FCC Over Possible ""Bill Shock"" Rules"
• 13:34 "Verizon big winner from having iPhone? Not so fast"
• 13:38 "Verizon President Talks iPhone Strategy"
• 13:43 "FACTBOX: AT&T and Verizon wage battle over iPhone"
• 13:43 "US stocks up on strong earnings [at Financial times]"
• 13:45 "Verizon Gets the iPhone"
• 13:45 "Verizon to start selling iPhone on Feb. 10"
• 13:49 "Apple: No LTE iPhone This Year?"
• 13:53 "AT&T: We're ""Evaluating"" The New iPhone Mobile Hotspot Feature"
• 13:58 "Apple: Surprise! Street Expresses Delight With VZ iPhone"
• 14:00 "[$$] Catching the Wave: Euro Rally Due"
• 14:02 "Did Apple disappoint with no 4G iPhone?"
• 14:03 "Mobile Payments: What Verizon's iPhone Means for Square"
• 14:04 "Verizon to Sell Apple IPad That Connects Directly to Its Network"
• 14:09 "Verizon Wireless ends long wait for iPhone fans"
• 14:10 "UPDATE 4-Verizon Wireless ends long wait for iPhone fans"
• 14:13 "Verizon Wireless ends long wait for iPhone fans"
• 14:25 "[audio] Before the Close from MarketWatch Radio Network [1.0 min]"
• 14:27 "Verizon Wireless ends long wait for iPhone fans"
• 14:27 "Verizon Wireless ends long wait for iPhone fans"
• 14:33 "Market Snapshot: U.S. stocks rise on earnings optimism"
• 14:33 "Verizon to start selling iPhone on Feb. 10"
• 14:41 "Analysts Go Out on Limb, Predict Verizon iPhone Will Be Big for Apple"
• 14:48 "Verizon Says You Can Switch If Santa Brought You a Droid"
• 14:49 "Verizon Wireless to Start IPhone Sales Next Month, Ending AT&T Exclusivity"
• 14:50 "Motorola: Opportunity In Lack Of LTE iPhone"
• 14:50 "Verizon iPhone, WoW: Hot Trends"
• 15:07 "Is Apple's iPhone a Scrooge?"
• 15:09 "How does the Verizon iPhone affect Android and AT&T?"
• 15:09 "More talk of smartphones diving into featurephone ground"
• 15:09 "Verizon iPhone unlimited data won't last forever"
• 15:09 "Why the Verizon 'dream phone' matters"
• 15:22 "[video] digits: Verizon Unveils Its iPhone [8.7 min]"
• 15:35 "With Verizon, Will Apple Take a Bite of More Enterprise?"
• 15:38 "5 'Dogs of the Dow' Worth Betting On"
• 15:38 "[video] iPhone: No wonders for Verizon stock"
• 15:44 "Why RadioShack Is Still Relevant"
• 15:50 "Apple: Jefferies Ups Tgt To $450; LTE iPhone In October?"
• 15:50 "BlackBerry PlayBook vs. Android Tablets"
• 15:51 "Happy Verizon iPhone Day!"
• 15:57 "[$$] Overheard"
• 15:58 "Verizon to start selling iPhone on Feb. 10"
• 16:12 "Verizon's Daniel Mead Doesn't Expect IPhone Price War: Video"
• 16:17 "Verizon to start selling iPhone on Feb. 10"
• 16:18 "Android to Lose From Verizon iPhone?"
• 16:20 "Investing for Income in 2011 with Stocks and Options Part 3: High-Yield Stocks"
• 16:24 "Tech Stocks: Techs post modest gains, but AMD sinks"
• 16:26 "[audio] Closing Bell from MarketWatch Radio Network [1.0 min]"
• 16:30 "Apple: Reasons For An LTE iPhone This Year"
• 16:30 "[video] digits: Verizon iPhone: What You Need to Know [4.1 min]"
• 16:30 "[video] digits:What Verizon iPhone Means for Apple, Google [2.1 min]"
• 16:31 "US STOCKS-Energy shares lift Wall St in light volume"
• 16:32 "Don't Buy The Verizon iPhone Yet, Unless You NEED A New Phone ASAP"
• 16:48 "[video] PM Report: China Stealth Fighter - Threat to U.S.? [10.2 min]"
• 16:50 "Stronger earnings reports push stocks higher"
• 17:50 "[video] Options Action"
• 17:54 "Stronger earnings reports push stocks higher"
• 17:55 "Factbox: AT&T and Verizon wage battle over iPhone"
• 18:09 "All Our iPhone Trades, Right Here & Now"
• 18:09 "Most active New York Stock Exchange-traded stocks"
• 18:44 "Doherty Sees 'Exodus' of AT&T IPhone Users to Verizon: Video"
• 19:00 "Lady Gaga Brags, U2 to Rival R.E.M. for 2011 Rock Crown: Preview"
• 19:40 "'Fast Money' Recap: Commodities Getting Frothy"
• 20:21 "MiFi Come, MiFi Go: Verizon Wireless To Sell iPads With 3G Connectivity"
• 22:02 "Tuesday ETF Roundup: UNG Surges on Weather, IYZ Tumbles on Increased Competition"
• 23:38 "Adding Up Parts at Goldman Sachs"

AT&T (T): Tuesday January 11th, 2011

• 00:01 "IPhone May Cost Verizon Wireless $5 Billion in Subsidies in First Year"
• 00:15 "ANALYSIS - AT&T faces tough year after losing iPhone exclusive"
• 02:47 "Telecoms' Dividend Yields"
• 03:03 "With iPhone Taunts, AT&T-Verizon Rivalry Escalates"
• 05:00 "[video] Verizon iPhone Data Plan"
• 05:57 "Cramer's Lightning Round - Viva Las Vegas (1/10/11)"
• 06:00 "Nielsen: Young People Across The Globe Love Their Cell Phones (But Use Them Differently)"
• 06:21 "[$$] RadioShack Needs Verizon Wireless"
• 06:45 "Half of Dow Companies to Post Profit Gains"
• 06:50 "Ahead of the Bell: Verizon iPhone"
• 07:15 "GM returning to Super Bowl for first time since '08."
• 07:27 "10 Things You Need To Know Before The Opening Bell"
• 07:27 "10 Things You Need To Know Before The Opening Bell"
• 07:32 "Lowenstein Says Verizon's Network Ready to Handle IPhone: Video"
• 07:40 "iPhone Raises Stakes of AT&T-Verizon Rivalry"
• 07:41 "10 Things You Need to Know Before the Opening Bell"
• 07:45 "Verizon big winner from having iPhone? Not so fast"
• 07:58 "AT&T and Verizon Trade Taunts Over iPhone"
• 08:03 "[video] 3 Stocks I Saw on TV"
• 08:05 "AT&T-Verizon rivalry kicks up a notch"
• 08:09 "10 Unusual Stocks Attracting Huge Interest This Morning"
• 08:22 "Will Verizon Demolish AT&T? It Did So Long Ago"
• 08:24 "The Big Questions About Verizon's IPhone"
• 08:24 "Verizon big winner from having iPhone? Not so fast"
• 08:25 "McCourt Says Verizon's Network Can Handle Apple's IPhone: Video"
• 08:36 "How Important Is Offering the iPhone on Verizon to Apple?"
• 08:45 "[$$] ETF Play of the Day: XLK"
• 09:02 "[video] AM Report: Chinese Stealth Jet Takes Flight [8.0 min]"
• 09:06 "Three Things to Watch For at Verizon iPhoneapalooza"
• 09:29 "Verizon iPhone Faces Even Bigger Foe"
• 09:33 "Trujillo Says 'Many Questions' Remain For Verizon IPhone: Video"
• 09:45 "Verizon IPhone Event Live Blog"
• 09:45 "[video] News Hub: Verizon iPhone: How will AT&T Respond? [2.7 min]"
• 09:52 "[video] Verizon's iPhone Buzz"
• 09:54 "Mobile Broadband Subs Seen Hitting 1 Billion In 2011"
• 09:58 "[audio] Opening Bell from MarketWatch Radio Network [1.0 min]"
• 10:03 "Video: Is the Verizon iPhone a Threat to Android?"
• 10:05 "Verizon iPhone expected to have unlimited data plan"
• 10:14 "[video] iPhone or Not, Verizon Is A Winner"
• 10:19 "Verizon iPhone: Live Blog"
• 10:37 "Verizon: Stay Tuned"
• 10:43 "Mobile Broadband Subscriptions to Surpass 1 Billion in 2011"
• 10:47 "LIVE: The Verizon iPhone Event"
• 10:54 "LIVE FROM NEW YORK: Verizon Gets the iPhone"
• 10:54 "Verizon iPhone: Will It Fix Verizon's $100 Billion Conundrum?"
• 11:10 "Verizon to Partner With Apple, Start Selling IPhone Next Month"
• 11:14 "Verizon and Apple Make It Official"
• 11:15 "Verizon Wireless to Start IPhone Sales Next Month, Ending AT&T Exclusivity"
• 11:15 "iPhone Breaks AT&T Marriage Today for Verizon"
• 11:18 "Verizon to launch iPhone 4 in February"
• 11:20 "Verizon iPhone: The basics"
• 11:24 "The Biggest Surprise About the Verizon iPhone: It's a Mobile Hotspot"
• 11:24 "Verizon to Carry Apple iPhone, Ending Months of Anticipation"
• 11:30 "Verizon Will Be Selling The iPhone Starting February 10th"
• 11:38 "UPDATE 2-Verizon Wireless to sell iPhone in February"
• 11:39 "PHOTOS: Verizon iPhone Has Different (Better?) Antenna Design Than AT&T iPhone"
• 11:39 "Verizon, AT&T, And Apple: Guess Which One's Stock Isn't Tanking Today"
• 11:43 "Verizon Wireless to Offer iPhone 4 in February"
• 11:45 "Verizon iPhone: 'It Begins'"
• 11:50 "Verizon to begin selling iPhone 4 in February"
• 11:51 "Stocks Led by Energy, Conglomerates"
• 11:51 "Verizon to sell Apple iPhone"
• 11:51 "Verizon: Began Discussions With Apple In 2008; Tim Cook Talks"
• 11:54 "Verizon Wireless to sell iPhone in February"
• 12:08 "Verizon iPhone Will Not Work Overseas"
• 12:13 "UPDATE 3-Verizon Wireless to sell iPhone in February"
• 12:16 "FACTBOX - AT&T and Verizon wage battle over iPhone"
• 12:17 "AT&T Preps for Verizon iPhone With New Ads, Price Cuts"
• 12:19 "Apple's Phil Schiller on Building the Verizon iPhone"
• 12:20 "AT&T Shrugs Off Verizon iPhone"
• 12:20 "COMPARISON: Verizon Vs. AT&T iPhone"
• 12:26 "FACTBOX - AT&T and Verizon wage battle over iPhone"
• 12:28 "Verizon iPhone: Yep, The Real Target is Google."
• 12:30 "AT&T: Five Reasons the Sky Won't Fall When iPhone Goes to Verizon"
• 12:35 "[audio] Midday Update from MarketWatch Radio Network [1.0 min]"
• 12:37 "FACTBOX - AT&T and Verizon wage battle over iPhone"
• 12:37 "Verizon Wireless to sell iPhone in February"
• 12:43 "Verizon to start selling iPhone on Feb. 10"
• 12:48 "Corrected: Factbox: AT&T and Verizon wage battle over iPhone"
• 12:59 "Verizon Wireless to sell iPhone in February"
• 13:05 "Verizon iPhone set for Feb. 10 release"
• 13:05 "Verizon to start selling iPhone Feb. 3"
• 13:07 "Moore Says 3 Million AT&T IPhone Users May Go to Verizon: Video"
• 13:11 "Cook Says Apple-Verizon Cooperation 'Just the Beginning': Video"
• 13:12 "COMPARISON: Verizon Vs. AT&T iPhone"
• 13:18 "Most active New York Stock Exchange-traded stocks"
• 13:18 "Summary Box: Verizon to start selling iPhone"
• 13:18 "[video] Apple's iPhone 4 Comes to Verizon [1.4 min]"
• 13:22 "Apple/Verizon Pt. 2: Should I stay or should I go now?"
• 13:23 "FACTBOX: AT&T and Verizon wage battle over iPhone"
• 13:23 "Halftime: Verizon iPhone To Drag Down Verizon Stock 10%?"
• 13:29 "Wireless Struggles Against FCC Over Possible ""Bill Shock"" Rules"
• 13:34 "Verizon big winner from having iPhone? Not so fast"
• 13:43 "FACTBOX: AT&T and Verizon wage battle over iPhone"
• 13:43 "US stocks up on strong earnings [at Financial times]"
• 13:45 "Verizon to start selling iPhone on Feb. 10"
• 13:49 "Apple: No LTE iPhone This Year?"
• 13:53 "AT&T: We're ""Evaluating"" The New iPhone Mobile Hotspot Feature"
• 13:58 "Apple: Surprise! Street Expresses Delight With VZ iPhone"
• 14:02 "Did Apple disappoint with no 4G iPhone?"
• 14:04 "Verizon to Sell Apple IPad That Connects Directly to Its Network"
• 14:05 "Verizon begins selling iPhone on Feb. 3"
• 14:05 "Verizon iPhone sales begin Feb. 3"
• 14:09 "Verizon Wireless ends long wait for iPhone fans"
• 14:10 "UPDATE 4-Verizon Wireless ends long wait for iPhone fans"
• 14:13 "Verizon Wireless ends long wait for iPhone fans"
• 14:25 "[audio] Before the Close from MarketWatch Radio Network [1.0 min]"
• 14:27 "Verizon Wireless ends long wait for iPhone fans"
• 14:27 "Verizon Wireless ends long wait for iPhone fans"
• 14:33 "Market Snapshot: U.S. stocks rise on earnings optimism"
• 14:33 "Verizon to start selling iPhone on Feb. 10"
• 14:41 "Analysts Go Out on Limb, Predict Verizon iPhone Will Be Big for Apple"
• 14:50 "Motorola: Opportunity In Lack Of LTE iPhone"
• 14:57 "An iPhone For Valentine's Day and iPad 2 Update"
• 14:58 Verizon
• 15:05 "Verizon iPhone sales begin Feb. 3"
• 15:05 "Verizon iPhone set for Feb. 10 release"
• 15:09 "How does the Verizon iPhone affect Android and AT&T?"
• 15:09 "More talk of smartphones diving into featurephone ground"
• 15:09 "Verizon iPhone unlimited data won't last forever"
• 15:09 "Why the Verizon 'dream phone' matters"
• 15:38 "5 'Dogs of the Dow' Worth Betting On"
• 15:38 "[video] iPhone: No wonders for Verizon stock"
• 15:50 "Apple: Jefferies Ups Tgt To $450; LTE iPhone In October?"
• 15:51 "Happy Verizon iPhone Day!"
• 15:58 "Verizon to start selling iPhone on Feb. 10"
• 16:06 "Verizon iPhone Is Not Mobile Messiah: Analyst"
• 16:10 "Stocks March Higher As Earnings Season Begins"
• 16:12 "Verizon's Daniel Mead Doesn't Expect IPhone Price War: Video"
• 16:17 "Verizon to start selling iPhone on Feb. 10"
• 16:18 "Android to Lose From Verizon iPhone?"
• 16:20 "Investing for Income in 2011 with Stocks and Options Part 3: High-Yield Stocks"
• 16:28 "Daley Has $7.7 Million of JPMorgan Stock to Divest on Way to White House"
• 16:30 "Apple: Reasons For An LTE iPhone This Year"
• 16:32 "Don't Buy The Verizon iPhone Yet, Unless You NEED A New Phone ASAP"
• 16:50 "Stronger earnings reports push stocks higher"
• 17:04 "Verizon Wireless ends long wait for iPhone fans"
• 17:08 "Verizon Wireless ends long wait for iPhone fans"
• 17:22 "Verizon Wireless ends long wait for iPhone fans"
• 17:22 "Verizon Wireless ends long wait for iPhone fans"
• 17:41 "California Reclaims 48,000 State-Owned Mobile Phones"
• 17:43 "[video] News Hub: Verizon iPhone - What's The Actual Cost? [3.0 min]"
• 17:54 "Stronger earnings reports push stocks higher"
• 17:55 "Factbox: AT&T and Verizon wage battle over iPhone"
• 18:09 "All Our iPhone Trades, Right Here & Now"
• 18:09 "Most active New York Stock Exchange-traded stocks"
• 18:44 "Doherty Sees 'Exodus' of AT&T IPhone Users to Verizon: Video"
• 20:21 "MiFi Come, MiFi Go: Verizon Wireless To Sell iPads With 3G Connectivity"
• 22:02 "Tuesday ETF Roundup: UNG Surges on Weather, IYZ Tumbles on Increased Competition"
• 23:38 "Adding Up Parts at Goldman Sachs"

Merck (MRK): Thursday January 13th, 2011

• 08:58 "JP Morgan Healthcare: Sanofi-Aventis, UCSF in Research Pact"
• 09:20 "Merck Statement on Changes to Clinical Studies of Vorapaxar"
• 09:41 "U.S. stocks open lower as joblessness rises"
• 09:41 "UPDATE 1-Merck stops giving clot drug in study, shares drop"
• 09:47 "Merck Drags on Stocks"
• 09:49 "Merck tumbles on drug study woes"
• 09:55 "Market Snapshot: U.S. stocks open lower as joblessness rises"
• 09:58 "Merck Blood-Thinner Study Ends in Stroke Patients, Limiting Possible Sales"
• 09:58 "Stocks Fall Slightly After Jobless Claims, PPI"
• 10:56 "Movers & Shakers: Thursday?s biggest gaining and declining stocks"
• 11:06 "Idaho in Drug Settlement With Warrick, Schering-Plough"
• 11:10 "Stocks Weighed by Healthcare Sector"
• 11:19 "[video] Markets Hub: US Stocks Fall As Merck Leads Decline [2.9 min]"
• 11:28 "FDA seeks less acetaminophen in prescription drugs"
• 11:34 "Commodities Look 'Very Attractive' Now: Kass"
• 11:41 "The Cheaper Way to Build the Best Portfolio"
• 11:43 "Biotech Stocks: Merck leads drug stocks lower"
• 11:43 "UPDATE 3-Merck clot drug seen unfit for stroke, shares fall"
• 11:47 "Merck Pipeline Setback: Big Pharma Losers"
• 11:49 "Merck clot drug seen unfit for stroke, shares fall"
• 11:53 "US STOCKS-Market flat, pares losses from jobless claims"
• 11:54 "Markets Going Nowhere After Weak US Data: Here Are The 10 Trades To Watch"
• 11:56 "Merck Learns There Is No Magic Pill"
• 11:56 "U.S. stocks bounce off lows; oil turns up"
• 12:16 "Bond Report: Treasurys rise after Fed purchases"
• 12:27 "Merck Shares Drop After Anti-Clotting Drug Trials Curtailed"
• 12:39 "Market Snapshot: U.S. stocks mixed as joblessness rises"
• 12:45 "Market flat, pares losses from jobless claims"
• 12:47 "A Painful Day for Merck"
• 13:05 "Merck tumbles on drug study update"
• 13:10 "US STOCKS-Market flat on weak jobless claims, Marathon's rise"
• 13:12 "Market flat, pares losses from jobless claims"
• 13:20 "Jobless claims weigh on US equities"
• 13:24 "Market flat on weak jobless claims, Marathon's rise"
• 13:29 "Dealpolitik: The Useful Corruption of Shareholder Lawsuits"
• 13:44 "Does Drug Setback Call Merck's Schering-Plough Purchase Into Question?"
• 13:47 "UPDATE 4-Merck clot drug seen unfit for stroke, shares fall"
• 13:48 "Market flat on weak jobless claims, Marathon's rise"
• 14:27 "Market Snapshot: Merck leads stocks lower"
• 14:35 "Merck Hurts, Stocks Stagger After Jobless Claims"
• 14:45 "Drug Study Torpedos Merck Stock"
• 14:52 "Bond Report: Treasurys lifted by Fed buys; 30-year sale in line"
• 14:55 "Jan. 13: Unusual Volume Leaders"
• 15:00 "Credit Suisse: All Isn't Lost for Merck"
• 15:12 "Merck weighs on Dow, market eyes earnings"
• 15:23 "This Trendy Pharma Is Making a Stylish Move"
• 15:36 "The Three Most Profitable Healthcare Trends of 2011"
• 15:47 "Market Snapshot: Stocks slide into close"
• 15:55 "Copying biotech medicine attracts more drugmakers"
• 16:00 "Stocks Slip Up On Awful US Data: Here's What You Need To Know"
• 16:08 "US STOCKS SNAPSHOT-Wall St dips on Merck, commodities"
• 16:09 "The Big Culprit In Today's DOW Drop"
• 16:09 "UPDATE 1-FDA seeks less acetaminophen in prescription drugs"
• 16:10 "U.S. stocks end lower as joblessness rises"
• 16:27 "US STOCKS-Merck, materials companies drag Wall St lower"
• 16:27 "[audio] Closing Bell from MarketWatch Radio Network [1.0 min]"
• 16:55 "Bond Report: Treasurys rise with jobless claims, auctions? end"
• 16:56 "[video] PM Report: Economists Turn Upbeat on Recovery [10.6 min]"
• 16:59 "[$$] Stocks in the Spotlight Thursday"
• 17:06 "US STOCKS-Merck, materials drag Wall St lower, Intel up late"
• 17:09 "[video] News Hub: Dow Dragged Down by Merck [1.3 min]"
• 17:11 "Market Snapshot: Stocks end lower on weak materials, drug shares"
• 18:29 "The Fed Takes Credit: Dave's Daily"
• 18:34 "[audio] After the Bell from MarketWatch Radio Network [1.0 min]"
• 18:39 "Trouble With Thrombin Inhibitor Takes a Toll on Merck Shares"
• 18:59 "Thursday ETF Roundup: UNG Sinks on Natural Gas Data, BLV Rises on Bond Boost"
• 20:06 "[$$] Options Traders Target Merck After Drug Study Halted"
• 20:26 "HK stocks seen lower on weak materials sector"

Bank of America (BAC): Friday January 14th, 2011

• 00:01 "Google, Renault, RBS, Stanford, Kissel, Madoff in Court News"
• 00:01 "Lehman, Madoff, FGIC, Chemtura, FairPoint, Quigley: Bankruptcy"
• 00:02 "Geithner starts talks on tax overhaul"
• 01:07 "Morgan Stanley's Gorman shakes up management"
• 01:25 "Morgan Stanley's Gorman shakes up management"
• 02:06 "Banks Are Poised to Pay Dividends After 3-Year Gap"
• 03:12 "UPDATE 1-India govt to decide on IOC share sale in 3-4 mths"
• 04:01 "What's Driving Earnings Higher?"
• 06:00 "5 Bank Stocks Under $5"
• 06:00 "5 Land Mines in Fourth Quarter Bank Earnings"
• 06:00 "[video] Hold, Don't Buy Bank Stocks: Fund Manager"
• 06:59 "Kissel Was Argumentative, Easily Upset, Sister-In-Law Testifies"
• 07:00 "Filling In FINRA's Gaps"
• 07:00 "General Electric Earnings: What to Expect"
• 07:00 "Worst-Performing Financial CEO of 2010"
• 07:16 "[$$] Earnings Scorecard: Commercial and Investment Banks"
• 07:18 "JPMorgan profit surges despite mortgage hit"
• 07:36 "JPMorgan's Fourth Quarter Profit Rises 47% to $4.8 Billion"
• 07:48 "RPT-UPDATE 4-Morgan Stanley's Gorman shakes up management"
• 07:59 "J.P. Morgan's Fourth-Quarter Profit Jumps 47%"
• 08:04 "Floods take major toll on Australian farms, mines"
• 08:08 "A Stock-Bond Hybrid That Flopped"
• 08:08 "Big Banks with Big Legal Bills"
• 08:08 "Home Depot's Fix-It Lady"
• 08:08 "Why the Private Sector Still Isn't Hiring"
• 08:12 "Citigroup, Bank of America Active Premarket After JPM Report"
• 08:19 "What's On: JPMorgan, Intel and Corporate Taxes"
• 08:24 "UPDATE 4-JPMorgan profit beats, helped by reserve release"
• 08:26 "Hybrid Debt, Materials Curbs, Tax Cheats: Compliance"
• 08:26 "JPMorgan profit beats, helped by reserve release"
• 08:37 "Deals of the Day: Groupon Talks IPO"
• 08:53 "GLOBAL MARKETS WEEKAHEAD-Reality checks ahead for investors"
• 08:54 "J.P. Morgan Earnings: Were They As Good as They Looked?"
• 08:55 "J.P. Morgan Earnings: Still Building Litigation Warchest"
• 09:01 "[$$] ETF Play of the Day: KBE"
• 09:10 "Foreclosure Wave Cresting, May Never Break"
• 09:40 "Earnings Jump At JPMorgan, But Street Stifled By China"
• 09:42 "U.S. Probes Banks' Mortgage Practices"
• 09:46 "JPMorgan profit rises 47 percent"
• 09:49 "J.P. Morgan: Mortgage Fights a 'Long, Ugly Mess'"
• 09:52 "ManTech International - Upgrades & Downgrades"
• 09:56 "Lbma Says Merrill Lynch Reclassified as Options Market Maker"
• 09:57 "UBS Upgrades Financials"
• 10:04 "American Capital prices share offer at $28 each"
• 10:09 "Schwab Takes High-Profile Stand Opposing Self-Regulatory Organization"
• 10:12 "BofA Suffers Online Glitches"
• 10:16 "JPMorgan: Strong Earnings, Muted Response"
• 10:23 "Stocks edge up after tepid inflation report"
• 10:33 "JPM Dividend: Jamie Dimon Balks"
• 10:34 "Your Index Fund Is Making a Bad Investment"
• 10:43 "Apple's Magic Share Price: $422"
• 10:45 "JPMorgan Chase Tops 4Q Estimates. Citigroup Is Up Next."
• 10:58 "Citigroup: Uncle Sam to Get Out, Completely"
• 11:01 "[$$] JPMorgan Chase Delivers Insight"
• 11:05 "Affiliated Managers gets $750M credit line"
• 11:08 "A Wave of Mergers And Acquisitions for Northeast And Mid-Atlantic Banks: An Exclusive Interview With Stephen M. Moss Of Janney Montgomery Scott LLC"
• 11:11 "UPDATE 6-JPMorgan profit rises 47 pct, beating estimates"
• 11:11 "[$$] Big Banks May Boost Dividends in 2011"
• 11:13 "New Zealand bourse winds up AXE trading venue"
• 11:20 "Greenspan, Fed Saw Threat of Stagflation After Katrina Devastation in 2005"
• 11:25 "A Wall Street Mystery: Where Was Goldman's CEO?"
• 11:29 "Goldman Lost an Extra $5B in Prop Trading: Report"
• 11:29 "JPMorgan profit rises 47 percent"
• 11:49 "[video] How JPMorgan's Quarter Affects Bank Stocks"
• 11:53 "Banks lead stocks higher, led by JPMorgan Chase"
• 12:03 "JPMorgan Earnings Lift Bank Stocks"
• 12:03 "John Paulson's Top Picks"
• 12:04 "CPI Shows Input-Cost Inflation Tempered By Overcapacity"
• 12:05 "Bank of America website having problems"
• 12:05 "JPMorgan Chase profit up 47%"
• 12:08 "Earnings schedule for week of 1/17/2011"
• 12:10 "CNNMoney stock market report -- Jan. 14, 2011"
• 12:26 "Banks lead stocks higher, led by JPMorgan Chase"
• 12:29 "J.P. Morgan?s profit jumps 47%"
• 12:36 "[$$] ETFs for the Dividend-Minded Investor"
• 13:44 "UPDATE 7-JPMorgan beats, sets bullish tone for bank earnings"
• 13:48 "AIG recap deal closes, focus moves to share sale"
• 13:48 "JPMorgan sets bullish tone for bank earnings"
• 13:50 "[video] J.P. Morgan Earnings Rise 47%"
• 13:52 "New Movie Revisits the Last 24 Hours of Lehman Brothers, or a Very Similar Firm"
• 13:52 "[$$] J.P. Morgan Gives Dow Modest Boost"
• 13:54 "Fed?s Tarullo Tells CNBC No Reason to Adjust Asset Purchases"
• 13:55 "Should You Buy 2010's Best Investments?"
• 14:00 "Banking On A Better 2011"
• 14:02 "[video] Rush Card"
• 14:05 "JPMorgan Credit Swaps Climb as Bank Results Rest on Reserves"
• 14:06 "How Celebrities Buy And Sell Homes"
• 14:07 "A female quota at Davos? Really?"
• 14:26 "JPMorgan Rises On Double Beat"
• 14:40 "Bank of America, Wells Fargo May Lose Under Plan to Simplify Mortgages"
• 14:43 "PNC, Capital One Advance After JPMorgan Posts Record Profit"
• 14:45 "Bank of America online banking down for some users"
• 14:45 "Banks take stock indexes higher"
• 14:56 "Financial Stocks: Financial shares higher after J.P. Morgan results"
• 15:01 "Halftime: Trading Financials Ahead of Earnings Next Week"
• 15:05 "Bank of America website having problems"
• 15:10 "Bank of America website suffers outage"
• 15:16 "AIG recap deal closes, focus moves to share sale"
• 15:21 "JPMorgan beats, sets bullish tone for bank earnings"
• 15:27 "PREVIEW-UPDATE 1-Volcker rule tests new US systemic risk council"
• 15:29 "Bank of America online banking down for some users"
• 15:37 "Bank of America Downplays Web Problems"
• 15:55 "Volcker rule tests new US systemic risk council"
• 15:57 "AIG recap deal closes, focus moves to share sale"
• 15:57 "Bank of America Website is Down For Some"
• 15:59 "[$$] Treasurys Fall, Giving Up Earlier Rally; Little Change for Week"
• 16:10 "Assured Guaranty was only muni bond insurer in 2010"
• 16:17 "U.S. stocks post seventh week of gains"
• 16:18 "Great Depression Part Deux Averted But Bubble Machine Still Blows"
• 16:18 "Volcker rule tests new US systemic risk council"
• 16:23 "Tax talks get good start, companies say"
• 16:29 "Banks take stock indexes higher, led by JPMorgan"
• 16:31 "Dow, S&P 500 Extend Seven-Week Winning Streak"
• 16:31 "[$$] Interesting Rotation in the Big Banks"
• 16:36 "Banks lead S&P 500 to seventh week of gains"
• 16:52 "Stocks Reach New Records"
• 16:54 "Market Snapshot: U.S. stocks rise to 30-month highs"
• 16:55 "Trading Radar: Key Earnings, China Headline the Week"
• 17:41 "JPMorgan beats, sets bullish tone for bank earnings"
• 18:15 "AIG recapitalization deal closes, share sale looms"
• 18:32 "Wall St Week Ahead: Investors crave more strong bank results"
• 18:46 "GLOBAL MARKETS WEEKAHEAD - Reality checks ahead for investors"
• 18:46 "US STOCKS - Banks lead S&P 500 to seventh week of gains"
• 18:46 "Volcker rule tests new US systemic risk council"
• 20:55 "AIG recapitalization deal closes, share sale looms"
• 20:55 "JPMorgan beats, sets bullish tone for bank earnings"

JP Morgan (JPM): Friday January 14th, 2011

• 07:29 "JPMorgan Profit Beats on Narrowing Loan Losses"
• 07:29 "JPMorgan reports jump in quarterly profit"
• 07:33 "US STOCKS-Futures fall, JPMorgan profit jumps"
• 07:34 "JPMorgan Net Rises 47% on Lower Credit Costs, Tops Estimates"
• 07:35 "Futures Lower Despite Earnings Beats"
• 07:35 "[audio] Before the bell from MarketWatch Radio Network [1.0 min]"
• 07:36 "JPMorgan's Fourth Quarter Profit Rises 47% to $4.8 Billion"
• 07:37 "J.P. Morgan posts quarterly profit of $4.8 billion"
• 07:39 "UPDATE 3-JPMorgan profit beats, helped by reserve release"
• 07:40 "The Good Leads"
• 07:41 "JPMorgan Chase's profit jumps 47 percent in 4Q"
• 07:41 "JPMorgan quarterly profits surge 47%"
• 07:42 "JPMorgan profit beats expectations"
• 07:42 "UPDATE 1-TABLE-JPMorgan fourth-quarter results"
• 07:45 "CNNMoney premarket report -- Jan. 14, 2011"
• 07:48 "RPT-UPDATE 4-Morgan Stanley's Gorman shakes up management"
• 07:56 "[$$] J.P. Morgan Profit Jumps 47%"
• 07:57 "Indications: U.S. stock futures point to soft open"
• 07:57 "Stock futures off, Dow, S&P seek 7th week of gains"
• 07:59 "J.P. Morgan's Fourth-Quarter Profit Jumps 47%"
• 08:26 "JPMorgan profit beats, helped by reserve release"
• 08:28 "JPMorgan Beats Street, Net Rises 47%"
• 08:28 "Morning Take-Out"
• 08:30 "Minyanville's T3 Morning Call: JPM, INTC Earnings Beats Can't Lift Futures"
• 08:30 "[$$] The Day Ahead: Follow the Leaders"
• 08:31 "JP Morgan results ahead of forecasts"
• 08:36 "[$$] It's All Baked In"
• 08:41 "Deals wrap: JPMorgan surprises"
• 08:41 "[$$] Futures Stay Weak After U.S. Data"
• 08:43 "M&T Beats Estimates"
• 08:44 "US STOCKS-Futures briefly tick lower after CPI, retail sales"
• 08:45 "Futures Hold Losses After CPI, Retail Sales"
• 08:46 "[$$] Will Intel Be the Downside Catalyst?"
• 08:52 "Live coverage of JPMorgan?s earnings call"
• 08:54 "J.P. Morgan Earnings: Were They As Good as They Looked?"
• 08:55 "J.P. Morgan Earnings: Still Building Litigation Warchest"
• 08:56 "JPMorgan Chase's profit jumps 47 percent in 4Q"
• 08:57 "JPMorgan reports jump in quarterly profit"
• 09:38 "CNNMoney stock market report -- Jan. 14, 2011"
• 09:38 "JPMorgan's Biggest Cost? Pay, Not Mortgages"
• 09:38 "UPDATE 5-JPMorgan profit rises 47 pct, beating estimates"
• 09:40 "Earnings Jump At JPMorgan, But Street Stifled By China"
• 09:41 "Financials open higher after J.P. Morgan earnings"
• 09:41 "Stocks Slip at Open After Mixed Economic News"
• 09:42 "U.S. Probes Banks' Mortgage Practices"
• 09:45 "Unique Convert Offering From Dendreon via JPMorgan"
• 09:46 "JPMorgan profit rises 47 percent"
• 09:47 "Friday Look Ahead: Intel, JPMorgan Earnings and Consumer Mood"
• 09:48 "J.P. Morgan?s profit jumps 47%"
• 09:49 "J.P. Morgan: Mortgage Fights a 'Long, Ugly Mess'"
• 09:50 "Stock Market Story: Jan. 14"
• 09:57 "JPMorgan Reports Q4 Results"
• 09:57 "Markets dip after JPMorgan earns, retail sales"
• 09:57 "UBS Upgrades Financials"
• 10:16 "JPMorgan: Strong Earnings, Muted Response"
• 10:23 "Stocks edge up after tepid inflation report"
• 10:24 "[$$] JPMorgan Worries Down the Drain"
• 10:25 "[$$] Mixed Start"
• 10:28 "US open: Dow trades flat"
• 10:30 "[$$] A Short-Term Retail Play"
• 10:31 "China's inflation move weighs on Wall St"
• 10:33 "JPM Dividend: Jamie Dimon Balks"
• 10:45 "[$$] Dip-Buyers' Delight"
• 10:47 "UPDATE 1-JPMorgan Q4 commods risk steady after market rally"
• 10:54 "Friday's ETF to Watch: Merrill Lynch Regional Bank HOLDR"
• 10:55 "J.P. Morgan banker pay slips -- to $369,651"
• 10:57 "Markets flat as JPMorgan results offset Intel"
• 10:58 "Citigroup: Uncle Sam to Get Out, Completely"
• 10:58 "Financial Stocks: Financial shares higher after J.P. Morgan results"
• 11:08 "A Wave of Mergers And Acquisitions for Northeast And Mid-Atlantic Banks: An Exclusive Interview With Stephen M. Moss Of Janney Montgomery Scott LLC"
• 11:11 "[$$] Big Banks May Boost Dividends in 2011"
• 11:16 "[$$] J.P. Morgan Bankers Earn Less"
• 11:20 "Greenspan, Fed Saw Threat of Stagflation After Katrina Devastation in 2005"
• 11:20 "JPMorgan Paces Bank Gains After Earnings"
• 11:25 "A Wall Street Mystery: Where Was Goldman's CEO?"
• 11:26 "Debit fee caps may hurt poorest customers: Dimon"
• 11:28 "Market Snapshot: Stocks edge up as J.P. Morgan lifts banks"
• 11:29 "Goldman Lost an Extra $5B in Prop Trading: Report"
• 11:29 "JPMorgan profit rises 47 percent"
• 11:29 "News Corp. in Talks to Buy Shine"
• 11:33 "JPM Earnings"
• 11:41 "Complacency, Fleeing Insiders Point to Earnings Season Drop"
• 11:42 "Gold Prices Tank on Moves to Curb Inflation"
• 11:42 "JPMorgan: Bonus Numbers Look Good But Bankers Are Wary"
• 11:46 "UPDATE 2-TABLE-JPMorgan fourth-quarter results"
• 11:49 "[video] How JPMorgan's Quarter Affects Bank Stocks"
• 11:53 "Banks lead stocks higher, led by JPMorgan Chase"
• 12:03 "JPMorgan Earnings Lift Bank Stocks"
• 12:05 "Bank of America website having problems"
• 12:05 "JPMorgan Chase profit soars 47%"
• 12:05 "JPMorgan Chase profit up 47%"
• 12:09 "A Buying Opportunity Before Stress-Test Round 2?"
• 12:10 "S&P 500 Is Poised to Drop; RBC, JPMorgan Differ on the Timing"
• 12:11 "[$$] Comeback After Dazzling Comeback"
• 12:12 "US STOCKS-Market rises as JPMorgan profit outweighs data"
• 12:26 "Banks lead stocks higher, led by JPMorgan Chase"
• 12:32 "[audio] Midday Update from MarketWatch Radio Network [1.0 min]"
• 12:39 "Hey, Jamie Dimon, You Sure About that Acronym?"
• 12:55 "Stocks hurt by global inflation concerns"
• 12:56 "China, U.S. Weigh on Europe's Markets"
• 13:10 "Market Snapshot: Stocks find footing J.P. Morgan lifts banks"
• 13:11 "US STOCKS-Wall St rises as JPMorgan earnings spur optimism"
• 13:14 "Spain, Portugal And A Virtuous Circle Of Bailouts"
• 13:20 "Most active New York Stock Exchange-traded stocks"
• 13:34 "4 Top Financials for 1Q from Bernstein Research"
• 13:36 "Bank Of America Website Down, 6 Hours And Counting"
• 13:43 "UPDATE 3-AIG recap deal closes, focus moves to share sale"
• 13:44 "JPMorgan Chase's income jumps 47 percent"
• 13:48 "AIG recap deal closes, focus moves to share sale"
• 13:48 "JPMorgan sets bullish tone for bank earnings"
• 13:50 "[video] J.P. Morgan Earnings Rise 47%"
• 13:52 "New Movie Revisits the Last 24 Hours of Lehman Brothers, or a Very Similar Firm"
• 13:53 "JPMorgan: JV Approval in China Positions Stock for Upside"
• 13:54 "Fed?s Tarullo Tells CNBC No Reason to Adjust Asset Purchases"
• 13:59 "Elisabeth Murdoch's Shine hires bank for review"
• 14:00 "Banking On A Better 2011"
• 14:03 "J P MORGAN CHASE & CO Files SEC form 8-K, Regulation FD Disclosure"
• 14:05 "JPMorgan Credit Swaps Climb as Bank Results Rest on Reserves"
• 14:06 "How Celebrities Buy And Sell Homes"
• 14:07 "A female quota at Davos? Really?"
• 14:10 "M&T Bank Tops Zacks Estimate"
• 14:17 "Summary Box: JPMorgan Chase's profit jumps in 4Q"
• 14:17 "What's Behind JP Morgan's $1.12 Shares"
• 14:20 "Hedge funds appeal ruling on WaMu trust securities"
• 14:26 "JPMorgan Rises On Double Beat"
• 14:31 "UK unmatched for bank bonus-bashing"
• 14:34 "VIX Falls to Three-Week Low After JPMorgan Posts Record Profit"
• 14:35 "US STOCKS-JPMorgan lifts Wall St, but banks due for pause"
• 14:40 "Bank of America, Wells Fargo May Lose Under Plan to Simplify Mortgages"
• 14:43 "PNC, Capital One Advance After JPMorgan Posts Record Profit"
• 14:45 "Banks take stock indexes higher"
• 14:50 "Explaining Confirmation"
• 14:58 "JPMorgan Chase Plans $3.25 Billion Debt Offering"
• 15:01 "Gold Prices Sink on Rising Inflation Concerns"
• 15:05 "Bank of America website having problems"
• 15:16 "AIG recap deal closes, focus moves to share sale"
• 15:17 "Bond Report: Treasurys fall as relief rally dissipates"
• 15:21 "JPMorgan beats, sets bullish tone for bank earnings"
• 15:22 "Market Snapshot: U.S. stocks rise as J.P. Morgan lifts banks"
• 15:27 "PREVIEW-UPDATE 1-Volcker rule tests new US systemic risk council"
• 15:39 "[$$] Crude Settles Above $91"
• 15:40 "Why Citi Won't Repeat JPMorgan's Success"
• 15:43 "What to Do With JP Morgan Now?"
• 15:50 "JPMorgan Chase Declares Preferred Stock Dividend"
• 16:15 "PNC, Capital One: Winners & Losers"
• 16:17 "U.S. stocks post seventh week of gains"
• 16:18 "Great Depression Part Deux Averted But Bubble Machine Still Blows"
• 16:18 "Volcker rule tests new US systemic risk council"
• 16:25 "[audio] Closing Bell from MarketWatch Radio Network [1.0 min]"
• 16:29 "Banks take stock indexes higher, led by JPMorgan"
• 16:30 "MarketWatch?s top 10 stories: Jan. 10-14"
• 16:31 "HCP, Kimco: REIT Broker Action"
• 16:31 "New Issue-JP Morgan sells $3.25 bln in 2 parts"
• 16:31 "US STOCKS-Banks lead S&P 500 to seventh week of gains"
• 16:31 "[$$] Interesting Rotation in the Big Banks"
• 16:32 "Coinstar, JPMorgan, Borders are big market movers"
• 16:48 "Stocks Buck Mixed Data to Head Higher"
• 16:50 "Buzz on the Street: Tech Stocks Continue to Lead the Way"
• 16:52 "Stocks Reach New Records"
• 16:54 "Market Snapshot: U.S. stocks rise to 30-month highs"
• 16:55 "[video] News Hub: Dow Ends at 2-1/2 Year High [1.5 min]"