I am having trouble with some basic IO operations using Clojure. I have a text file which I need to read, split with the "|" character, and enter into a list for later processing. Here are the contents of my text file:
1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533
And here is my current code:
((defn -main []
(println "Enter an option: \n")
(let [choice (read-line)]
(cond (= choice "1")
(let [cust-contents (slurp "file.txt")
nums-as-strings (clojure.string/split cust-contents #"|")
numbers (map read-string nums-as-strings)]
(print numbers)
)
)
) ) )
(-main)
I would think this code to work, however here is the error I get when running my program:
(; Execution error at user/eval7923$-main (REPL:11).
; EOF while reading
Could anyone please guide me on where I went wrong and on how to fix this?
That error is actually caused when you call read-string with argument " ".
And where that " " comes from? You used the wrong regex in clojure.string/split, you should use #"\|" instead of #"|".
And even then, you can't call read-string on everything, as it would soon crash again, trying to parse "456-4567".
Also, ending parentheses belong to the same line.
If your file contains newlines, you will need clojure.string/split-lines and if you will also implement 2 Display Product Table, 3. Display Sales Table and so on (I know the context for this code), case will be better than cond.
Here is your code with some improvements:
(ns homework.core
(:require [clojure.string :as s])
(:gen-class))
(defn -main []
(println "Enter an option: \n")
(let [choice (read-line)]
(case choice
"1" (->> (s/split-lines (slurp "file.txt"))
(mapv #(s/split % #"\|"))
println)
"Wrong number")))
(-main)
For one thing, you should not have 2 left parentheses in a row like ((defn ....
Also, CSV parsing has many libraries to remove the drudgery of reinventing the wheel. My favorite is illustrated below:
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test)
(:require
[tupelo.csv :as csv]))
(verify
; ***** NOTE ***** most CSV has a header line, which is added below for convenience!
(let [data-str "id|name|address|phone
1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533 "
entity-maps (csv/csv->entities data-str {:separator \|})]
(is= entity-maps
[{:address "123 Here Street", :id "1", :name "John Smith", :phone "456-4567"}
{:address "43 Rose Court Street", :id "2", :name "Sue Jones", :phone "345-7867"}
{:address "165 Happy Lane", :id "3", :name "Fan Yuhong", :phone "345-4533"}])))
This unit test leaves out the file-reading part as that just complicates things in a unit test.
The above solution is based on my favorite template project, which also has a list of documentation sources toward the end.
You can find examine the source code for tupelo.csv
and also the unit tests.
To clarify your code, here is a cleaned-up version:
(verify
(let [data-str "1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533 "
data-lines (str/split-lines data-str)
token-table-str (vec (for [line data-lines]
(let [token-strs (str/split line #"\|")]
(mapv str/trim token-strs))))]
(is= token-table-str
[["1" "John Smith" "123 Here Street" "456-4567"]
["2" "Sue Jones" "43 Rose Court Street" "345-7867"]
["3" "Fan Yuhong" "165 Happy Lane" "345-4533"]])))
Notice that I didn't parse the data past the String stage.
Add [clojure-csv/clojure-csv "2.0.1"] to your lein dependencies.
Do lein deps to install it.
Put into a file e.g. "/home/me/test.csv"
1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533
(ns csvread
(:require [clojure-csv.core :as csv]))
(defn parse-table [file]
(csv/parse-csv (slurp file) :delimiter \|))
(defn parse-table-from-string [s]
(csv/parse-csv s :delimiter \|))
Apply it:
(parse-table "/home/me/test.csv")
;; => (["1" "John Smith" "123 Here Street" "456-4567"]
;; ["2" "Sue Jones" "43 Rose Court Street" "345-7867"]
;; ["3" "Fan Yuhong" "165 Happy Lane" "345-4533 "])
(def s "1|John Smith|123 Here Street|456-4567
2|Sue Jones|43 Rose Court Street|345-7867
3|Fan Yuhong|165 Happy Lane|345-4533")
(parse-table-from-string s)
Related
This example is a little contrived. The goal is to create a macro that loops over some values and programmatically generates some code.
A common pattern in Python is to initialize the properties of an object at calling time as follows:
(defclass hair [foo bar]
(defn __init__ [self]
(setv self.foo foo)
(setv self.bar bar)))
This correctly translates with hy2py to
class hair(foo, bar):
def __init__(self):
self.foo = foo
self.bar = bar
return None
I know there are Python approaches to this problem including attr.ib and dataclasses. But as a simplified learning exercise I wanted to approach this with a macro.
This is my non-working example:
(defmacro self-set [&rest args]
(for [[name val] args]
`(setv (. self (read-str ~name)) ~val)))
(defn fur [foo bar]
(defn __init__ [self]
(self-set [["foo" foo] ["bar" bar]])))
But this doesn't expand to the original pattern. hy2py shows:
from hy.core.language import name
from hy import HyExpression, HySymbol
import hy
def _hy_anon_var_1(hyx_XampersandXname, *args):
for [name, val] in args:
HyExpression([] + [HySymbol('setv')] + [HyExpression([] + [HySymbol
('.')] + [HySymbol('self')] + [HyExpression([] + [HySymbol(
'read-str')] + [name])])] + [val])
hy.macros.macro('self-set')(_hy_anon_var_1)
def fur(foo, bar):
def __init__(self, foo, bar):
return None
Wbat am I doing wrong?
for forms always return None. So, your loop is constructing the (setv ...) forms you request and then throwing them away. Instead, try lfor, which returns a list of results, or gfor, which returns a generator. Note also in the below example that I use do to group the generated forms together, and I've moved a ~ so that the read-str happens at compile-time, as it must in order for . to work.
(defmacro self-set [&rest args]
`(do ~#(gfor
[name val] args
`(setv (. self ~(read-str name)) ~val))))
(defclass hair []
(defn __init__ [self]
(self-set ["foo" 1] ["bar" 2])))
(setv h (hair))
(print h.bar) ; 2
I'm new to Clojure and I have a question regarding nested doseq loops.
I would like to iterate through a sequence and get a subsequence, and then get some keys to apply a function over all the sequence elements.
The given sequence has an structure more or less like this, but with hundreds of books, shelves and many libraries:
([:state/libraries {6 #:library {:name "MUNICIPAL LIBRARY OF X" :id 6
:shelves {3 #:shelf {:name "GREEN SHELF" :id 3 :books
{45 #:book {:id 45 :name "NECRONOMICON" :pages {...},
{89 #:book {:id 89 :name "HOLY BIBLE" :pages {...}}}}}}}}])
Here is my code:
(defn my-function [] (let [conn (d/connect (-> my-system :config :datomic-uri))]
(doseq [library-seq (read-string (slurp "given-sequence.edn"))]
(doseq [shelves-seq (val library-seq)]
(library/create-shelf conn {:id (:shelf/id (val shelves-seq))
:name (:shelf/name (val shelves-seq))})
(doseq [books-seq (:shelf/books (val shelves-seq))]
(library/create-book conn (:shelf/id (val shelves-seq)) {:id (:book/id (val books-seq))
:name (:book/name (val books-seq))})
)))))
The thing is that I want to get rid of that nested doseq mess but I don't know what would be the best approach, since in each iteration keys change. Using recur? reduce? Maybe I am thinking about this completely the wrong way?
Like Carcigenicate says in the comments, presuming that the library/... functions are only side effecting, you can just write this in a single doseq.
(defn my-function []
(let [conn (d/connect (-> my-system :config :datomic-uri))]
(doseq [library-seq (read-string (slurp "given-sequence.edn"))
shelves-seq (val library-seq)
:let [_ (library/create-shelf conn
{:id (:shelf/id (val shelves-seq))
:name (:shelf/name (val shelves-seq))})]
books-seq (:shelf/books (val shelves-seq))]
(library/create-book conn
(:shelf/id (val shelves-seq))
{:id (:book/id (val books-seq))
:name (:book/name (val books-seq))}))))
I would separate "connecting to the db" from "slurping a file" from "writing to the db" though. Together with some destructuring I'd end up with something more like:
(defn write-to-the-db [conn given-sequence]
(doseq [library-seq given-sequence
shelves-seq (val library-seq)
:let [{shelf-id :shelf/id,
shelf-name :shelf/name
books :shelf/books} (val shelves-seq)
_ (library/create-shelf conn {:id shelf-id, :name shelf-name})]
{book-id :book/id, book-name :book/name} books]
(library/create-book conn shelf-id {:id book-id, :name book-name})))
I am currently experimenting with om and try to load external data for displaying it in a component.
My detail component:
(defn detail-component [app owner opts]
(reify
om/IInitState
(init-state [_]
(om/transact! app [:data] (fn [] "Test")))
om/IWillMount
(will-mount [_]
(go (let [foo (<! (fetch-something 1))]
(om/update! app #(assoc % :data foo))
)))
om/IRender
(render [_]
(dom/div nil
(dom/h1 nil "Foo")
(om/build another-component app)
)
))
)
(fetch-something is retrieving data from an API).
(defn another-component [{:keys [data]}]
(om/component
(.log js/console data)
(dom/h2 nil "Another component goes here")
(dom/h2 nil (data :description))
)
)
So to summarize, detail-component fetches data before mounting, attaches it to app and builds another-component. another-component then takes the description out of that data and displays it.
However when executing, I am getting Uncaught TypeError: Cannot read property 'call' of null at the point where I am trying to access the description. This indicates to me that at the point when another-component is getting built, the data is not there yet and it fails.
How can I tell om to build the app when data is available? Or do I have to build in some nil? checks?
Working example using state:
project.clj
(defproject asajax "0.0.1-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License - v 1.0"
:url "http://www.eclipse.org/legal/epl-v10.html"
:distribution :repo}
:min-lein-version "2.3.4"
:source-paths ["src/clj" "src/cljs"]
:dependencies [[org.clojure/clojure "1.6.0"]
[org.clojure/clojurescript "0.0-2371"]
[org.clojure/core.async "0.1.267.0-0d7780-alpha"]
[om "0.7.3"]
[com.facebook/react "0.11.2"]]
:plugins [[lein-cljsbuild "1.0.4-SNAPSHOT"]]
:hooks [leiningen.cljsbuild]
:cljsbuild
{:builds {:asajax
{:source-paths ["src/cljs"]
:compiler
{:output-to "dev-resources/public/js/asajax.js"
:optimizations :whitespace
:pretty-print true}}}})
core.cljs
(ns asajax.core
(:require [om.core :as om :include-macros true]
[om.dom :as dom :include-macros true]
[cljs.core.async :as async])
(:require-macros [cljs.core.async.macros :refer (go)]))
(enable-console-print!)
(def app-state (atom {}))
(defn another-component [{:keys [data]}]
(reify
om/IRenderState
(render-state [_ state]
(dom/div nil
;; (.log js/console data)
(dom/h2 nil "Another component goes here")
(dom/h2 nil (:description state))))))
(defn fetch-something [x]
(let [c (async/chan)]
(go
;; pretend a blocking call
;; wait for 2 sec
(<! (async/timeout 2000))
(>! c {:data "Roast peach & Parma ham salad"
:description "This is a lovely light starter with fantastic sweet, salty and creamy flavours"}))
c))
(defn detail-component [app owner opts]
(reify
om/IInitState
(init-state [_]
{:data "Test"})
om/IWillMount
(will-mount [_]
(go (let [foo (<! (fetch-something 1))]
;; (prn "GOT" foo)
(om/set-state! owner foo))))
om/IRenderState
(render-state [_ state]
(dom/div nil
(dom/h1 nil (:data state))
(om/build another-component app
{:state (om/get-state owner)})))))
(om/root
detail-component
app-state
{:target (. js/document (getElementById "app"))})
UPDATE2
Here is one using sablono and not using set-state!
Add to project.clj
[sablono "0.2.22"]
Full core.cljs
(ns asajax.core
(:require [om.core :as om :include-macros true]
[om.dom :as dom :include-macros true]
[cljs.core.async :as async]
[sablono.core :as html :refer-macros [html]])
(:require-macros [cljs.core.async.macros :refer (go)]))
(enable-console-print!)
(def app-state (atom {:data "Initial"
:description "Loading..."}))
(defn another-component [{:keys [description]}]
(om/component
(html
[:.description
[:h2 "Another component"]
[:h2 description]])))
(defn fetch-something [x]
(let [c (async/chan)]
(go
;; pretend a blocking call
;; wait for 2 sec
(<! (async/timeout 2000))
(>! c {:data "Roast peach & Parma ham salad"
:description "This is a lovely light starter with fantastic sweet, salty and creamy flavours"}))
c))
(defn detail-component [app owner opts]
(reify
om/IWillMount
(will-mount [_]
(go (let [foo (<! (fetch-something 1))]
;; (prn "GOT" foo)
(om/update! app foo))))
om/IRender
(render [_]
(html [:div
[:h1 (:data app)]
(om/build another-component app)
]))))
(om/root
detail-component
app-state
{:target (. js/document (getElementById "app"))})
My .emacs:
;; enable orgmode en set files
(require 'org-install)
(setq org-directory "~/Dropbox/GTD/")
(add-to-list 'auto-mode-alist '("\\.org$" . org-mode))
(define-key global-map "\C-cl" 'org-store-link)
(define-key global-map "\C-ca" 'org-agenda)
(setq org-log-done t)
(setq org-agenda-files (list (concat org-directory "nextactions.org")
(concat org-directory "projects.org")
(concat org-directory "birthday.org")))
;; Daily action list
(setq org-agenda-custom-commands
'(
("D" "Daily Action List"
(
(agenda "" ((org-agenda-ndays 1)
(org-agenda-sorting-strategy
(quote ((agenda time-up priority-down tag-up) )))
(org-deadline-warning-days 0)
))))
;; Office list
("H" "Office and Home Lists"
((agenda)
(tags-todo "OFFICE")
(tags-todo "HOME")))
)
)
;; Turn on diary within org-mode
(setq org-agenda-include-diary t)
;; Turn on Capture
(setq org-default-notes-file (concat org-directory "notes.org"))
(define-key global-map "\C-cc" 'org-capture)
;; Capture templates
(setq org-capture-templates
'(
("t" "Todo" entry (file+headline (concat org-directory "nextactions.org") "Inbox") "* TODO %?\n %i\n %a")
("j" "Journal" entry (file+datetree (concat org-directory "journal.org")) "* %?\nEntered on %U\n %i\n %a")
)
)
C-c c presents the capture menu buffer. I press then t and capture the buffer that appears (CAPTURE-notes.org). After C-c C-c the entry is added to notes.org instead of nextactions.org section "Inbox".
I have no .emacs parsing errors, how can I fix this so the todo capture-template puts its entry in nextactions.org?
Edit: Setting org-default-notes-file to nextactions.org lets me operate at least the first org-capture template (because its file is the same anyway). The second one stays writing to the default-notes-file.
What org-mode version are you using? I don't remember the exact version, but quoted capture templates were not eval'ed before that.
So you should try either
Update org-mode
use backquotes
(setq org-capture-templates
`(("t" "Todo" entry (file+headline ,(concat org-directory "nextactions.org")
"Inbox")
"* TODO %?\n %i\n %a")
("j" "Journal" entry (file+datetree ,(concat org-directory "journal.org"))
"* %?\nEntered on %U\n %i\n %a")
))
use the literal filepath
(setq org-capture-templates
'(("t" "Todo" entry (file+headline "~/Dropbox/GTD/nextactions.org"
"Inbox")
"* TODO %?\n %i\n %a")
("j" "Journal" entry (file+datetree "~/Dropbox/GTD/journal.org")
"* %?\nEntered on %U\n %i\n %a")
))
Apart from that I see no problem with your config, I use a similar one.
infact you can even refile from the capture buffer to which ever agenda file you want to with C-c C-w.
How to insert a blob in database using the clojure.contrib.sql?
I've tried the following reading from a file but I'm getting this exception:
SQLException:
Message: Invalid column type
SQLState: 99999
Error Code: 17004
java.lang.Exception: transaction rolled back: Invalid column type (repl-1:125)
(clojure.contrib.sql/with-connection
db
(clojure.contrib.sql/transaction
(clojure.contrib.sql/insert-values :test_blob [:blob_id :a_blob] [3 (FileInputStream. "c:/somefile.xls")]) ))
Thanks.
I was able to solve this by converting the FileInputStream into a ByteArray.
(clojure.contrib.sql/with-connection
db
(clojure.contrib.sql/transaction
(clojure.contrib.sql/insert-values :test_blob [:blob_id :a_blob] [3 (to-byte-array(FileInputStream. "c:/somefile.xls"))]) ))
In theory, you can use any of the clojure.contrib.sql/insert-* methods to insert a blob, passing the blob as either a byte array, java.sql.Blob or a java.io.InputStream object. In practice, it is driver-dependent.
For many JDBC implementations, all of the above work as expected, but if you're using sqlitejdbc 0.5.6 from Clojars, you'll find your blob coerced to a string via toString(). All the clojure.contrib.sql/insert-* commands are issued via clojure.contrib.sql/do-prepared, which calls setObject() on a java.sql.PreparedStatement. The sqlitejdbc implementation does not handle setObject() for any of the blob data types, but defaults to coercing them to a string. Here's a work-around that enables you to store blobs in SQLite:
(use '[clojure.contrib.io :only (input-stream to-byte-array)])
(require '[clojure.contrib.sql :as sql])
(defn my-do-prepared
"Executes an (optionally parameterized) SQL prepared statement on the
open database connection. Each param-group is a seq of values for all of
the parameters. This is a modified version of clojure.contrib.sql/do-prepared
with special handling of byte arrays."
[sql & param-groups]
(with-open [stmt (.prepareStatement (sql/connection) sql)]
(doseq [param-group param-groups]
(doseq [[index value] (map vector (iterate inc 1) param-group)]
(if (= (class value) (class (to-byte-array "")))
(.setBytes stmt index value)
(.setObject stmt index value)))
(.addBatch stmt))
(sql/transaction
(seq (.executeBatch stmt)))))
(defn my-load-blob [filename]
(let [blob (to-byte-array (input-stream filename))]
(sql/with-connection db
(my-do-prepared "insert into mytable (blob_column) values (?)" [blob]))))
A more recent reply to this thread with the code to read the data as well :
(ns com.my-items.polypheme.db.demo
(:require
[clojure.java.io :as io]
[clojure.java.jdbc :as sql]))
(def db {:dbtype "postgresql"
:dbname "my_db_name"
:host "my_server"
:user "user",
:password "user"})
(defn file->bytes [file]
(with-open [xin (io/input-stream file)
xout (java.io.ByteArrayOutputStream.)]
(io/copy xin xout)
(.toByteArray xout)))
(defn insert-image [db-config file]
(let [bytes (file->bytes file)]
(sql/with-db-transaction [conn db-config]
(sql/insert! conn :image {:name (.getName file) :data bytes}))))
(insert-image db (io/file "resources" "my_nice_picture.JPG"))
;;read data
(defn read-image [db-config id]
(-> (sql/get-by-id db-config :image id)
:data
(#(new java.io.ByteArrayInputStream %))))
I believe it's just the same way you'd insert any other value: use one of insert-records, insert-rows or insert-values. E.g.:
(insert-values :mytable [:id :blobcolumn] [42 blob])
More examples: http://github.com/richhickey/clojure-contrib/blob/master/src/test/clojure/clojure/contrib/test_sql.clj