Using Clojure's Datomic APi, I have an architecture where I'd like to pass in a transact function, to be executed. However, trying to call the passed in transact function doesn't work. The repl recognizes it as a Symbol. And it evaluates, but no data is committed, and no future is returned, meaning, there's no returned transaction ID.
However, directly calling (datomic.api/transact conn [adatom]), works as expected. How can I make the abouve scenario work?
(defn some-fn[conn mapped-fn adatom]
(datomic.api/transact conn [adatom]) ;; works
#_(mapped-fn conn [adatom]) ;; does not work - evaluates, but no data committed, no future returned, meaning, no returned transaction ID
)
Thanks
It's not very clear what you are trying to do. For example:
why not call d/transact inside some-fn?
is mapped-fn a function that will be applied to many facts (and in this case it is d/transact)?
what is an overall intent?
To just blindly follow your example, it does work and returns a "future":
user=> (use '[datomic.api :only (db) :as d])
nil
user=> (d/create-database "datomic:mem://unsure-db")
false
user=> (def c (d/connect "datomic:mem://unsure-db"))
#'user/c
user=> (defn f [conn mapped-fn fact] (mapped-fn conn fact))
#'user/f
user=> (f c d/transact [])
#<promise$settable_future$reify__4526#2da07336: {:db-before datomic.db.Db#8835fddc, :db-after datomic.db.Db#6e2a2e78, :tx-data [#Datum{:e 13194139534313 :a 50 :v #inst "2013-09-03T15:23:34.977-00:00" :tx 13194139534313 :added true}], :tempids {}}>
Make sure you have a valid connection (e.g. you are connected [to the right database]), and the database is there.
Related
In my schema I have the attribute :base/type that is supposed to exist for every entity created. To check that this is indeed true, I'm trying to find entities where it is missing:
[:find [?entities ...]
:in $ :where
[(missing? $ ?entities :base/type)]]
Unfortunately this gives me back:
Execution error (Exceptions$IllegalArgumentExceptionInfo) at datomic.error/arg (error.clj:57).
:db.error/insufficient-binding [?entities] not bound in expression clause: [(missing? $ ?entities :base/type)]
How should this query be constructed?
This is because your query is too general. You need at least one positive clause in where statement if you use query API. You can access raw index to get the result, though. If you have enough RAM, you can:
(def snapshot (d/db connection)) ; your db snapshot
(def all-datoms (d/datoms snapshot :eavt))
(def base-type-id (:db/id (d/entity snapshot :base/type))
(def entities (group-by #(.e %) all-datoms))
(def entities-without-base-type (map
(comp #(into {} %) (partial d/entity snapshot) first)
(filter (fn [[k l]]
(empty? (filter #(= base-type-id (.a %))
l)))
entities)))
(def only-relevant-entities (filter #(not (or (:db/ident %) (:db/txInstant %))) entities-without-base-type))
only-relevant-entities
The last filter is to get rid of attribute definitions and transactions (they are stored as datoms in the db as well!).
If you have too many entities you can chunk datoms using the async atoms API.
Using ::singer/songs as an example attribute, this is how to do the query:
[:find [?entities ...]
:in $ :where
[?entities ::singer/songs ?s]
[(missing? $ ?entities :base/type)]]
Unfortunately (in my answer) many such queries would be required until the whole database has been covered. So another query with ::song/composers, etc...
Often I need load huge data size from database server. Sometimes it have million rows and more. So I try to download data lazily. That what I want to do: I want get a lazy-sequence and pull data partial from server, i.e if row count is more than 500, I want primarily get with help of that lazy-sequence first 500 elements, then by another request i want receive next 500 elements and so on until i recieve all data from server.
But I have a problem. Clojure jdbc realize entire lazy-sequence, but I want obtain data from it partially.
I research that question and find good reply about similar problem:
clojure.java.jdbc lazy query
I took this example and wrote this:
(defn get_data
[arg1 arg2]
(let [full-db-spec (get ...)
sql_query (get ...)
row-n (atom 0)
prepared-statement (-> full-db-spec
(jdbc/get-connection)
(jdbc/prepare-statement sql_query {:fetch-size 3}))]
(jdbc/with-db-transaction [tx full-db-spec]
(jdbc/query full-db-spec [prepared-statement arg1 arg2]
{:fetch-size 3
:row-fn (fn [r] (do (prn "r" #row-n) (swap! row-n inc) r))
:result-set-fn identity}))))
Here I want to get a lazy-sequence to further extract data partially from this lazy-sequence. But when :result-set-fn is contain identity or (take 500 ...) the code return error: The result set is closed. Why? But when I change :result-set-fn to first or doall or last It works fine but it realize full lazy-sequence!
I use:
ms sql [com.microsoft.sqlserver/mssql-jdbc "6.3.3.jre8-preview"] (yet I test it on postgresql [org.postgresql/postgresql "9.4.1212.jre7"]. Same result)
[org.clojure/java.jdbc "0.7.3"]
That lazy sequence is reading values from your connection, but the connection is closed outside of the with-db-transaction scope. You need to realize/do further processing inside of that scope.
In the official docs for Datomic (http://docs.datomic.com/schema.html) under the heading 'Creating new partitions' it says that a new partition (communities) can be created like this:
{:db/id #db/id[:db.part/db]
:db/ident :communities}
Here the ':communities' is not written as 'db.part/communities'
I can not install a new partition this way. For me it has to be with the leading 'db.part/'. Is the documentation wrong, or am I not seeing the bigger picture?
If you read further in the documentation, you'll see that you're missing another datom required for that transaction (labeled with "Here is the complete transaction..."). That datom is (with user assigned tempid as -1 optional):
[:db/add :db.part/db :db.install/partition #db/id[:db.part/db -1]]
Anything transacted with a tempid that resolves to the system partition (:db.part/db) must also include a datom marking the installation, as with :db.install/partition and :db.install/attribute (the reverse ref version for attribute included in the map is more common).
Transacting the full example from the docs works fine:
(def tx [{:db/id #db/id[:db.part/db -1]
:db/ident :communities}
[:db/add :db.part/db :db.install/partition #db/id[:db.part/db -1]]])
#(d/transact conn tx)
;; returns successful tx map
I have a schema like this:
:user/first-name
:user/last-name
:user/password
:user/groups
;; assume there are x more ident
:user/email
I'm pulling this by doing:
(d/pull db '[*] some-id)
Apparently this will also pull :user/password which I don't want, and doing this is cumbersome:
(d/pull db
[:user/first-name :user/last-name ;; all fields except :user/password]
some-id)
Is there anyway to do something like this:
(d/pull db ['* (except :user/password)] some-id)
You can use datomic's filter function to filter out datoms you don't need from db value.
In your case something like
(d/pull (filter db #(not (= (second %2) :user/password))) some-id)
What is a query to list all partitions of a Datomic database?
This should return
[[:db.part/db] [:db.part/tx] [:db.part/user] .... ]
where .... is all of the user defined partitions.
You should be able to get a list of all partitions in the database by searching for all entities associated with the :db.part/db entity via the :db.install/partition attribute:
(ns myns
(:require [datomic.api :as d]))
(defn get-partitions [db]
(d/q '[:find ?ident :where [:db.part/db :db.install/partition ?p]
[?p :db/ident ?ident]]
db))
Note
The current version of Datomic (build 0.8.3524) has a shortcoming such that :db.part/tx and :db.part/user (two of the three built-in partitions) are treated specially and aren't actually associated with :db.part/db via :db.install/partition, so the result of the above query function won't include the two.
This problem is going to be addressed in one of the future builds of Datomic. In the meantime, you should take care of including :db.part/tx and :db.part/user in the result set yourself.
1st method - using query
=> (q '[:find ?i :where
[:db.part/db :db.install/partition ?p] [?p :db/ident ?i]]
(db conn))
2nd method - from db object
(filter #(instance? datomic.db.Partition %) (:elements (db conn)))
The second method returns sequence of datomic.db.Partition objects which may be useful if we want to get additional info about the partition.
Both methods have known bug/inconsistency: they don't return :db.part/tx and :db.part/user built-in partitions.