Scala stock charts, part 2 - Getting the quotes

I am Swedish so I am for the moment only interested in the Swedish stock market. I have found a site where to get the quotes, it is in text format and I intend to read this information and save it in in other files, one file for each instrument (I call on single stock instrument since it is a form of financial instrument).

The format of the daily file is: (There is an example file to work with at : http://sites.google.com/site/ironicprogrammer/files/20110817k.txt)

Slutkurser från OMX 2011-08-17 Slutkurser från Burgundy 2011-08-17
Ticker Aktie +/- +/-% Köp Sälj Senast Högst Lägst Oms (antal) Oms (SEK) Senast Högst Lägst Oms (antal) Oms (SEK)
OMX Stockholm Large Cap
ABB ABB Ltd -2,30 -1,66 136,40 136,50 136,40 139,40 136,40 3299500 453921900 136,6 139,4 136,6 183000 25240000

It is hard to see in that little box, however. first row is title information of the file.
Slutkurser från OMX 2011-08-17[6 tabs]Slutkurser från Burgundy 2011-08-17

That is not really necessary and can be skipped as the next line, but the next line is interesting to see the structure of the quotes:

Ticker Aktie +/- +/-% Köp Sälj Senast Högst Lägst Oms (antal) Oms (SEK)[6 tabs]Senast Högst Lägst Oms (antal) Oms (SEK)

 When reading the file I will only be interested in the quotes from OMX, the Burgundy I do not want so when saving the quotes to different files I will not consider these, so it is the blue marked fileds that would be used.

The third row

OMX Stockholm Large Cap

is the name of the first list of quotes. It continues like this in the file;
Listname
Ticker Aktie ...
...
Listname ...

Ticker Aktie ...
...

I am only interested in the first three lists, the forth is named "Externa listan", and I will use this name to break the read.

So lets start write som code (finally!!)

I use the REPL for a little bit trial before I put in in classes/objects.

To test to read the file:

scala> val url = "http://sites.google.com/site/ironicprogrammer/files/" url: java.lang.String = http://sites.google.com/site/ironicprogrammer/files/ scala> val file = "20110817k.txt" file: java.lang.String = 20110817k.txt scala> import scala.io.Source import scala.io.Source scala> val fc = Source.fromURL(url+file) fc: scala.io.BufferedSource = non-empty iterator scala>; fc.getLines foreach { println } Slutkurser fr?n OMX 2011-08-17 Slutkurser fr?n Burgundy 2011-08-17 Ticker Aktie +/- +/-% K?p S?lj Senast H?gst L?gst Oms (antal) Oms (SEK) Senast H?gst L?gsOms (antal) Oms (SEK)<br /> OMX Stockholm Large Cap ABB ABB Ltd -2,30 -1,66 136,40 136,50 136,40 139,40 136,40 3299500 453921900 136,6 139,4 136,6 183000 25240000<br /> ALFA Alfa Laval -0,80 -0,68 117,50 117,60 117,50 119,10 114,80 1121500 131304100 117,6 118,9 114,49223 5760000

Ok, everything seams to work so this need to be taken car of in some other way.

Lets put it in a bit more organized way:

I put a file in src/main/scala/ironic that is named QuoteLoad.scala with the following content:

QuoteLoad.scala

/*
 @author Peter Johansson / Ironic programmer 
 */

package scala.ironic

// To be able to handle Iterator[QuoteItem]
trait QuoteItem 

case class InvalidQuote(date: String, content:String) extends QuoteItem
case class QuoteList(name:String) extends QuoteItem
case class Quote(date: String, ticker :String, aktie : String,
    diffKr : Double, diffPerc : Double, buy : Double,
    sell : Double, last : Double, high : Double,
    low : Double, returnNo : Double, returnKr : Double
  ) extends QuoteItem

// Singleton to handle loading of quotes
object QuoteLoad {
   def main(args: Array[String]) = {
      
      loadQuotes("http://sites.google.com/site/ironicprogrammer/files/","20110817")
   }
   
   val QuoteExtractorRE = """([[^Ticker]^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t([^\t]+)\t.*""".r
   val ListExtractorRE = """(OMX[^\t]+)\t+""".r
  
   def loadQuotes(url: String, date: String){
      import scala.io.Source
      
      val uri = url + date + "k.txt"
      val fc = Source.fromURL(uri) 

      val instruments = processLines(fc.getLines(),date)
      println (instruments)
      instruments foreach { inst => inst match {
        case ilist: QuoteList => 
            println(ilist)
        case q: Quote =>
            println(q)
        case _ => println(inst)//
        }
      }
   }
  
    // Should go through the lines from the file and make a iterable with QuoteItem's. 
    def processLines(lines: Iterator[String],date: String) : Iterator[QuoteItem] = {
      val instruments = lines takeWhile(s=> ! s.startsWith("Externa listan")) 
      instruments map { inst =>  
         inst match {
          case ListExtractorRE(list) => QuoteList(list)//"{\"instrumentList\": \"" + list + "\"}"
   case QuoteExtractorRE(ticker,aktie,diffKr,diffPerc,buy,sell,last,high,low,returnNo,returnKr) 
            => Quote(date,ticker,aktie,diffKr.replace(",",".").toDouble,diffPerc.replace(",",".").toDouble,buy.replace(",",".").toDouble,sell.replace(",",".").toDouble,last.replace(",",".").toDouble,high.replace(",",".").toDouble,low.replace(",",".").toDouble,returnNo.replace(",",".").toDouble,returnKr.replace(",",".").toDouble) 
          case _ => InvalidQuote(date,inst.replace("\t","[tab]"))
        } 
      } 
    }
    
}

Ok there may be some features to explain here. 
First of all, I make use of an singleton object here so I will be able to call from a terminal window. So it has a main method, but I also extract the functionality so the future client will be able to use the same methods to load quotes.

There is a trait so I can avoid getting Iterator[Any] and instead getting Iterator[QuoteItem] to be able to handle the case class objects only, when I later will save the quotes to different files.

QuoteList: is for holding the name of a list (there is different list of quotes according to its category, that is more or less the amount of trading that is done and the size of the companies)
Quote: is for holding the quote for the day for a specific share
InvalidQuote: is for holding (and logging for error checking) an invalid row from the file.

There is two regexp's that is handled later in pattern matching to extract the lists and quotes from the file. They look for the structure in the file for valid quotes and the simpler one is for extracting the list name that is only one text.

def loadQuotes(url: String, date: String): handles the load from the URL and calls a helper method to extract the valid quotes and lists.

This will have to do for a first version, later I will save these quotes to different files.