I use F# once every few months or so and in between it seems I forget everything, so I hope you'll excuse my ignorance. My code below is pulling data from Yahoo. It's a good example that represents what I need to do. The first row returned has the column headings. I need to take the data (the tail of the list) and insert it into the database. What's the best way to generate an insert statement based on the column headings returned (the column headings match the database column names)?
In the example below dataWithHeaders.[0] will contain "Date,Open,High,Low,Close,Volume,Adj Close." Should I just take that string and put brackets around the headings to create the insert? Then in insertData add the value as a parameter? Is there a more elegant solution?
let url = System.String.Format("http://ichart.finance.yahoo.com/table.csv?s={0}&g=d&ignore=.csv", "FB")
let splitLineIntoArray (line : string) =
line.Split(",".ToCharArray())
let insertData (data : string[]) =
// insert data
()
let client = new WebClient()
let dataWithHeaders =
client.DownloadString(url).Split(Environment.NewLine.ToCharArray())
let data =
dataWithHeaders
|> Array.toList
|> List.tail
|> List.map(splitLineIntoArray)
|> List.iter insertData
If you're loading the data into SQL Server you can use this excellent CSV reader (free) and the SqlBulkCopy class. It's simple and efficient.
let loadStockPrices ticker =
use client = new WebClient()
let url = sprintf "http://ichart.finance.yahoo.com/table.csv?s=%s&g=d&ignore=.csv" ticker
use stringReader = new StringReader(client.DownloadString(url))
use csvReader = new CsvReader(stringReader, hasHeaders=true)
use con = new SqlConnection("<connection_string>")
con.Open()
use bulkCopy = new SqlBulkCopy(con, DestinationTableName="<destination_table>")
bulkCopy.WriteToServer(csvReader)
The destination table should have the same columns as the incoming data (OHLC, etc).
EDIT: Type Providers might be a good way to go, but SqlBulkCopy is def. known for its simplicity.
Type Provider Code for Insert: http://msdn.microsoft.com/en-us/library/hh361033(v=vs.110).aspx#BKMK_UpdateDB
type dbSchema = SqlDataConnection<"Data Source=MYSERVER\INSTANCE;Initial Catalog=MyDatabase;Integrated Security=SSPI;">
let db = dbSchema.GetDataContext()
// Enable the logging of database activity to the console.
db.DataContext.Log <- System.Console.Out
let newRecord = new dbSchema.ServiceTypes.Table1(Id = 100,
TestData1 = 35,
TestData2 = 2.0,
Name = "Testing123")
let newValues =
[ for i in [1 .. 10] ->
new dbSchema.ServiceTypes.Table3(Id = 700 + i,
Name = "Testing" + i.ToString(),
Data = i) ]
// Insert the new data into the database.
db.Table1.InsertOnSubmit(newRecord)
db.Table3.InsertAllOnSubmit(newValues)
try
db.DataContext.SubmitChanges()
printfn "Successfully inserted new rows."
with
| exn -> printfn "Exception:\n%s" exn.Message
I did something similar. Actually this code I wrote while watching Luca Bolognese give a presentation on F#. This actually will scrape yahoo's feed and return the standard dev. and variance on stock prices.
Full project here: https://github.com/djohnsonm/Stock-Ticker-App
open System.Net
open System.IO
let internal loadPrices ticker = async {
let url = #"http://ichart.finance.yahoo.com/table.csv?s=" + ticker + "&d=6&e=22&f=2011&g=d&a=2&b=13&c=1986&ignore=.csv"
let req = WebRequest.Create(url)
let resp = req.GetResponse()
let stream = resp.GetResponseStream()
let reader = new StreamReader(stream)
let csv = reader.ReadToEnd()
let prices =
csv.Split([|'\n'|])
|> Seq.skip 1
|> Seq.map (fun line -> line.Split([|','|]))
|> Seq.filter(fun values -> values |> Seq.length = 7)
|> Seq.map(fun values ->
System.DateTime.Parse(values.[0]),
float values.[6])
return prices}
type StockAnalyzer (lprices, days) =
let prices =
lprices
|> Seq.map snd
|> Seq.take days
static member GetAnalyzers (tickers, days) =
tickers
|> Seq.map loadPrices
|> Async.Parallel
|> Async.RunSynchronously
|> Seq.map (fun prices -> new StockAnalyzer(prices, days))
member s.Return =
let lastPrice = prices |> Seq.nth 0
let startPrice = prices |> Seq.nth(days-1)
lastPrice / startPrice - 1.
member s.StdDev =
let logRets =
prices
|> Seq.pairwise
|> Seq.map (fun (x,y) -> log(x/y))
let mean = logRets |> Seq.average
let sqr x = x * x
let var = logRets |> Seq.averageBy (fun r -> sqr (r-mean))
sqrt var
Related
I have created my own type in F# called Accounts and I have then created objects for each account.
type Account() =
let AccountNumber = ""
let mutable Balance:float = 0.0
Every account has two fields, AccountNumber (string) and Balance (float).
I have then created an object for every account that holds the AccountName and the Balance.
let acc1 = new Account()
acc1.Insert("John",10.0)
let acc2 = new Account()
acc2.Insert("Mike",50.0)
How do I create a list that holds each account (object)? I have tried the following:
let AccountList : Account list = [acc1; acc2 ; acc3; acc4 ; acc5; acc6]
let AccountList : Account obj list = [acc1; acc2 ; acc3; acc4 ; acc5; acc6]
I cannot solve the problem using the above method because I have to create two sequences from the list:
Sequence 1: All accounts with a balance greater or equal to zero and less than 50
Sequence 2: All accounts with a balance above 50
How do I create a list of my custom type in F# and how do I create two sequences of that list?
It is not clear what exactly are you struggling with. However, the following simple example should illustrate most of the key ideas that you probably need to use. First, here is a small version of your Account class (note that I would normally use an immutable record, but I kept it the way you did it):
type Account(balance:float) =
let mutable balance = balance
member x.Balance = balance
member x.Add(difference) =
balance <- balance + difference
I do not see what issue you have with creating the list. The following works just fine:
let acc1 = Account(100.0)
let acc2 = Account(10.0)
let accountList = [acc1; acc2]
Now, to answer the question about finding accounts with balance over 50, you can use the List.filter function to create a new filtered list:
let above50 =
accountList |> List.filter (fun acc ->
acc.Balance > 50.0)
EDIT If you wanted to use a record instead, then you would define the type as:
type Account = { Balance : float }
And create a value using:
let acc1 = { Balance = 100.0 }
So I created this answer for the other one but I was waiting on the comment to see if I would answer. And the homework like aspect of this :)
So if you have criteria that bucket an account and want to do that in a single pass, you might want to look at groupBy. Here I use a boolean because there are only 2 possibilities but numbers or a discriminated union are good candidates.
open System
type Account(accountNumber:string, startingBalance:Int64) =
let mutable balance = startingBalance
member _.Balance = balance
member _.Deposit amount = balance <- balance + amount
member _.Withdraw amount = balance <- balance - amount
override _.ToString() = accountNumber
let allAccounts = [Account("ABC1", 10L); Account("ABC2", 50L)]
let grouped = allAccounts |> List.groupBy (fun a -> a.Balance >= 50L) |> Map.ofList
let under50 = grouped |> Map.tryFind false |> Option.defaultValue []
let overIncl50 = grouped |> Map.tryFind true |> Option.defaultValue []
printfn "Under: %A" under50
printfn "Over: %A" overIncl50
I have a data contract (WCF) with a field defined as:
[<DataContract(Namespace = _Namespace.ws)>]
type CommitRequest =
{
// Excluded for brevity
...
[<field: DataMember(Name="ExcludeList", IsRequired=false) >]
ExcludeList : int array option
}
I want to from the entries in the ExcludeList, create a comma separated string (to reduce the number of network hops to the database to update the status). I have tried the following 2 approaches, neither of which create the desired string, both are empty:
// Logic to determine if we need to execute this block works correctly
try
// Use F# concat
let strList = request.ExcludeList.Value |> Array.map string
let idString = String.concat ",", strList
// Next try using .NET Join
let idList = String.Join ((",", (request.ExcludeList.Value.Select (fun f -> f)).Distinct).ToString ())
with | ex ->
...
Both compile and execute but neither give me anything in the string. Would greatly appreciate someone pointing out what I am doing wrong here.
let intoArray : int array option = Some [| 1; 23; 16 |]
let strList = intoArray.Value |> Array.map string
let idString = String.concat "," strList // don't need comma between params
// Next try using .NET Join
let idList = System.String.Join (",", strList) // that also works
Output:
>
val intoArray : int array option = Some [|1; 23; 16|]
val strList : string [] = [|"1"; "23"; "16"|]
val idString : string = "1,23,16"
val idList : string = "1,23,16"
I am new to programming and F# is my first language.
Here is part of my code:
let splitArrayIntoGroups (inputArray: string[]) (groupSize: int) =
let groups = new LinkedList<string[]>()
let rec splitRecursively currentStartIndex currentEndIndex =
groups.AddLast(inputArray.[currentStartIndex..currentEndIndex]) |> ignore
let newEndIndex = Math.Min((inputArray.Length - 1), (currentEndIndex + groupSize))
if newEndIndex <> currentEndIndex then
splitRecursively (currentStartIndex + groupSize) newEndIndex
splitRecursively 0 (groupSize - 1)
groups
I want this function to be able to accept arrays of any type (including types that I define myself) as input. What changes should I make?
This was already answered but here you have an implementation not using a linked list but just an array of lists
let rec split<'T> (input: 'T array) size =
let rec loopOn (tail : 'T array) grouped =
let lastIndex = Array.length tail - 1
let endindx = min (size - 1) lastIndex
let arrWrapper = (fun e -> [|e|])
let newGroup = tail.[0..endindx]
|> List.ofArray
|> arrWrapper
|> Array.append grouped
match tail with
| [||] -> newGroup
|> Array.filter (fun e -> List.length e > 0)
| _ -> loopOn tail.[endindx + 1..] newGroup
let initialState = [|List.empty<'T>|]
loopOn input initialState
Because this is generic implementation you can call it with different types
type Custom = {Value : int}
let r = split<int> [|1..1000|] 10
let r2 = split<float> [|1.0..1000.0|] 10
let r3 = split<Custom> [|for i in 1..1000 ->
{Value = i}|] 10
replace string[] with _[] in the function signature.
I need to write a running number - at the beginning of existing files. All I have been able to do so far is to write to the end of each file. Any help please?
How about (warning! untested code):
open System.IO
File.ReadAllText("c:\\old.txt")
|> fun s -> let r = new StreamReader(s)
let data = r.ReadToEnd
r.Close()
s.Close()
data
let writeStringToFile d =
(File.OpenWrite("c:\\new.txt"), d)
|> fun ((s : FileStream), (d : string)) -> let r = new StreamWriter(s)
r.WriteLine("new data")
r.Write(d)
r.Close()
s.Close()
getStringFromFile()
|> writeStringToFile
EDIT:
Much better solution:
File.WriteAllText("c:\\new.txt", "new Text" + File.ReadAllText("c:\\old.txt"))
I recently asked this question:
Replay Recorded Data Stream in F# and combined that code with a subset of the functionality I found here:
http://www.mattssoftwareblog.com/?p=271
which combined looks like this:
#r "System.Reactive"
#r "System.CoreEx"
#r "FSharp.PowerPack"
#r "WindowsBase"
#r "PresentationCore"
#r "PresentationFramework"
#r "System.Xaml"
#r "System.Interactive.dll"
open System
open System.Linq
open System.Collections.Generic
open System.Net
open System.IO
open System.Threading
open System.Windows
open System.Windows.Input
open System.Windows.Controls
open System.Windows.Shapes
open System.Windows.Media
open System.Xaml
I need to use the events generated here (which came from my earlier SO question):
let prices = [ (0, 10.0); (1000, 10.5); (500, 9.5); (2500, 8.5); (500, 10.0); (1000, 10.5); (500, 9.5); (2500, 8.5) ]
let evt = new Event<float>()
async { for delay, price in prices do
do! Async.Sleep(delay)
evt.Trigger(price) }
|> Async.StartImmediate
evt.Publish.Add(printfn "Price updated: %A")
to use as the data source for the line which is randomly created here (the code below comes from the blog article I mentioned):
let create f =
Observable.Create<_>(fun x ->
f x
new System.Action((fun () -> ())))
let fromEvent (event:IEvent<_,_>) = create (fun x -> event.Add x.OnNext)
// Random Walker
let rand = Random()
let mutable m = 0.
let randomWalker() =
m <- m + (rand.NextDouble() * 10.) - 5.
m
let timer = new System.Timers.Timer()
timer.Interval <- 100.
let timerObs = (timer.Elapsed |> fromEvent).Select(fun _ -> randomWalker())
let chartWindow = new Window(Height = 600., Width = 600.)
let canvas = new Canvas()
chartWindow.Content <- canvas
chartWindow.Show()
let line xs =
let segs =
seq { for x , y in xs |> List.tail ->
LineSegment(Point(x,y), true) :> PathSegment }
let (sx, sy) = xs |> List.head
PathGeometry([PathFigure(Point(sx,sy), segs, false)])
let plot xs (path:Path) =
let now = DateTime.Now
let timeSpan = TimeSpan(0,1,0)
let width = 600.
let height = 600.
let pts = xs |> List.map (fun (x:Timestamped<float>) ->
(600.-(now - (x.Timestamp.DateTime)).TotalMilliseconds * 600. / timeSpan.TotalMilliseconds),x.Value + 300.)
path.Dispatcher.BeginInvoke(new SendOrPostCallback(fun pts -> path.Data <- line (pts :?> (float*float)list)), pts) |> ignore
let trailing (timespan:TimeSpan) (obs:IObservable<'
a>) =
obs.Timestamp()
.Scan([], fun ys x ->
let now = DateTime.Now
let timespan = timespan
x :: (ys |> List.filter (fun x -> (now - x.Timestamp.DateTime) < timespan)))
.Where(fun xs -> xs |> List.length > 1)
// Main Path
let mainPath = new Path(Stroke=Brushes.Blue, StrokeThickness=1.)
canvas.Children.Add(mainPath)
let trailingRandomsSub = (timerObs |> trailing (TimeSpan.FromSeconds(60.))).Subscribe(fun xs -> plot xs mainPath)
timer.Start()
If you paste this into an interactive session you will see a blue line emerge which is generated randomly and not using my new evt Event. I guess my confusion is not understanding how to make and use an Observable from my evt. Basically, how can I make evt my data source for the blue line?
Thanks in advance,
Bob
In F#, the IEvent<'T> interface inherits from IObservable<'T>. This means that you can use F# events in any place where an observable is expected.
The last bit of your application (that takes the event, adds time stamps, uses Scan to get lists containing the items generated so far and plots the progress) can be written like this:
let trailingRandomsSub =
evt.Publish.Timestamp()
|> Observable.scan (fun l e -> e::l) []
|> Observable.add (fun xs -> plot xs mainPath)
F# provides wrappers for some of the Rx functions, so you can use Observable.scan, which has a bit more F#-friendly syntax. Observable.add is just another syntax for Subscribe.
The key difference between F# events and observables is that observables start when you attach a handler. On the other hand, the F# event that you create using Async.StartImmediate starts immediately when the StartImmediate method is called (this means - to get the sample working, you need to evaluate everything at once, or write a function that starts the event).