Related
I'm likely missing something simple here, so I apologize in advance. I am also aware that there is likely a better approach to this, so I'm open to that as well.
I'm trying to run a PowerShell script that will look at an array of values, comparing them to see the value of the difference between two elements of an array.
Below is a sample data set I'm using to test with that is imported into powershell from CSV:
1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.7, 1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.7, 2.9, 3.0
What I'm trying to accomplish is running through this list and comparing the second entry with the first, the third with the second, fourth with the third, etc, adding the element to $export ONLY if it has a value that is at least 0.2 greater than the previous element.
Here's what I've tried:
$import = get-content C:/pathtoCSVfile
$count = $import.Length-1;
$current=0;
Do
{
$current=$current+1;
$previous=$current-1
if (($import[$current]-$import[$previous]) -ge 0.2)
{
$export=$export+$import[$current]+"`r`n";
}
}
until ($current -eq $count)
Now I've run this with Trace on and it assigns values to $current and $previous and runs the subtraction of the two as described in the if condition on each loop through, but ONLY for the value of 2.7 ($import[14]-$import[13]) is it registering that the if condition has been met, thus leaving only a single value of 2.7 in $export. I expected other values (1.7, 1.9, and 2.9) to also be added to the $export variable.
Again, this is probably something stupid/obvious I'm overlooking, but I can't seem to figure it out. Thanks in advance for any insight you can offer.
The problem is that decimal fractions have no exact representation in the implicitly used [double] data type, resulting in rounding errors that cause your -ge 0.2 comparison to yield unexpected results.
A simple example with [double] values, which are what PowerShell implicitly uses with number literals that have a decimal point:
PS> 2.7 - 2.5 -ge 0.2
True # OK, but only accidentally so, due to the specific input numbers.
PS> 1.7 - 1.5 -ge 0.2
False # !! Due to the inexact internally binary [double] representation.
If you force your calculations to use the [decimal] type instead, the problem goes away.
Applied to the above example (appending d to a number literal in PowerShell makes it a [decimal]):
PS> 1.7d - 1.5d -ge 0.2d
True # OK - Comparison is now exact, due to [decimal] values.
Applied in the context of a more PowerShell-idiomatic reformulation of your code:
# Sample input; note that floating-point number literals such as 1.0 default to [double]
# Similarly, performing arithmetic on *strings* that look like floating-point numbers
# defaults to [double], and Import-Csv always creates string properties.
$numbers = 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.7, 1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.7, 2.9, 3.0
# Collect those array elements that are >= 0.2 than their preceding element
# in output *array* $exports.
$exports = foreach ($ndx in 1..($numbers.Count - 1)) {
if ([decimal] $numbers[$ndx] - [decimal] $numbers[$ndx-1] -ge 0.2d) {
$numbers[$ndx]
}
}
# Output the result array.
# To create a multi-line string representation, use $exports -join "`r`n"
$exports
The above yields:
1.7
1.9
2.7
2.9
I am trying to create the curve fit with scipy for the energy eigenvalues calculated from a 4x4 Hamiltonian matrix. In the following error "energies" corresponds to the function in which I define the Hamiltonian, "xdata" is an array given after and out of the function and corresponds to k and "e" is the energy eigenvalues that a get.
The error seems to be at the Hamiltonian matrix. However if I run the code without the curve_fit everything works fine.
I have also tried using np.array according to other questions I found here but again it doesn't work.
If a give a specific xdata in the curve fit, like xdata[0], the code works but it doesn't help me much since I want the fit using all values.
Does anyone know what is the problem? Thank you all in advance!
Traceback (most recent call last):
File "fitest.py", line 70, in <module>
popt, pcov = curve_fit(energies,xdata, e)#,
File "/eb/software/Python/2.7.12-intel-2016b/lib/python2.7/site-packages/scipy/optimize/minpack.py", line 651, in curve_fit
res = leastsq(func, p0, args=args, full_output=1, **kwargs)
File "/eb/software/Python/2.7.12-intel-2016b/lib/python2.7/site-packages/scipy/optimize/minpack.py", line 377, in leastsq
shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
File "/eb/software/Python/2.7.12-intel-2016b/lib/python2.7/site-packages/scipy/optimize/minpack.py", line 26, in _check_func
res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
File "/eb/software/Python/2.7.12-intel-2016b/lib/python2.7/site-packages/scipy/optimize/minpack.py", line 453, in _general_function
return function(xdata, *params) - ydata
File "fitest.py", line 23, in energies
[ 0.0, 0.0, 0.0, ep-2*V4*cos(kpt*d) ]],dtype=complex)
TypeError: only length-1 arrays can be converted to Python scalars
Code:
from numpy import sin, cos, array
from scipy.optimize import curve_fit
from numpy import *
from numpy.linalg import *
def energies(kpt, a=1.0, b=2.0, c=3.0, f=4.0):
e1=-15.0
e2=-10.0
d=1.0
v0=(-2.0/d**2)
V1=a*v0
V2=b*v0
V3=c*v0
V4=d*v0
basis=('|S, s>', '|S,px>', '|S, py>', '|S,pz>')
h=array([[ e1-2*V1*cos(kpt*d), -2*V2*1j*sin(kpt*d), 0.0, 0.0 ],
[ 2*V2*1j*sin(kpt*d), e2-2*V3*cos(kpt*d), 0.0, 0.0],
[ 0.0, 0.0, e2-2*V4*cos(kpt*d), 0.0],
[ 0.0, 0.0, 0.0, e2-2*V4*cos(kpt*d) ]],dtype=complex)
e,psi=eigh(h)
return e
print energies(kpt=0.0)
k2=0.4*2*pi/2.05
print energies(kpt=k2)
xdata = array([0.0,k2])
print xdata
popt, pcov = curve_fit(energies, xdata, e)
print " "
print popt
print " "
Your problem has nothing to do with your fit, you run into the same problem, if you perform
print energies(xdata)
The reason for this error message is that you put an array kpt into h as an array element and then tell numpy, to transform this array kpt into a complex number. Numpy is kind enough to transform an array of length one into a scalar, which then can be transformed into a complex number. This explains, why you didn't get an error message with xdata[0]. You can easily reproduce your problem like this
import numpy as np
#all fine with an array of length one
xa = np.asarray([1])
a = np.asarray([[xa, 2, 3], [4, 5, 6]])
print a
print a.astype(complex)
#can't apply dtype = complex to an array with two elements
xb = np.asarray([1, 2])
b = np.asarray([[xb, 2, 3], [4, 5, 6]])
print b
print b.astype(complex)
Idk, what you were trying to achieve with your energies function, so I can only speculate, what you were aiming at, when constructing the h array. Maybe a 3D array like this?
kpt = np.asarray([1, 2, 3])
h = np.zeros(16 * len(kpt), dtype = complex).reshape(len(kpt), 4, 4)
h[:, 0, 0] = 2 * kpt + 1
h[:, 0, 1] = kpt ** 2
h[:, 3, 2] = np.sin(kpt)
print h
I am looking at Accelerate to compute mean and standard deviation of arrays in Swift.
I can do the mean. How do I do the standard deviation?
let rr: [Double] = [ 18.0, 21.0, 41.0, 42.0, 48.0, 50.0, 55.0, 90.0 ]
var mn: Double = 0.0
vDSP_meanvD(rr, 1, &mn, vDSP_Length(rr.count))
print(mn) // prints correct mean as 45.6250
// Standard Deviation should be 22.3155
You can compute the standard deviation from the mean value and
the mean square value (compare https://en.wikipedia.org/wiki/Standard_deviation#Identities_and_mathematical_properties and https://en.wikipedia.org/wiki/Algebraic_formula_for_the_variance):
import Accelerate
let rr: [Double] = [ 18.0, 21.0, 41.0, 42.0, 48.0, 50.0, 55.0, 90.0 ]
var mn: Double = 0.0 // mean value
vDSP_meanvD(rr, 1, &mn, vDSP_Length(rr.count))
var ms: Double = 0.0 // mean square value
vDSP_measqvD(rr, 1, &ms, vDSP_Length(rr.count))
let sddev = sqrt(ms - mn * mn) * sqrt(Double(rr.count)/Double(rr.count - 1))
print(mn, sddev)
// 45.625 22.315513501982
Alternatively (for iOS 9.0 and later or macOS 10.11 and later), use vDSP_normalizeD:
var mn = 0.0
var sddev = 0.0
vDSP_normalizeD(rr, 1, nil, 1, &mn, &sddev, vDSP_Length(rr.count))
sddev *= sqrt(Double(rr.count)/Double(rr.count - 1))
print(mn, sddev)
// 45.625 22.315513501982
an add-on for #Martin R's answer: There is also a vDSP_normalize function for Float/single precision.
func vDSP_normalize(UnsafePointer<Float>, vDSP_Stride, UnsafeMutablePointer<Float>?, vDSP_Stride, UnsafeMutablePointer<Float>, UnsafeMutablePointer<Float>, vDSP_Length)
//Compute mean and standard deviation and then calculate new elements to have a zero mean and a unit standard deviation. Single precision.
func vDSP_normalizeD(UnsafePointer<Double>, vDSP_Stride, UnsafeMutablePointer<Double>?, vDSP_Stride, UnsafeMutablePointer<Double>, UnsafeMutablePointer<Double>, vDSP_Length)
//Compute mean and standard deviation and then calculate new elements to have a zero mean and a unit standard deviation. Double precision.
I am trying to create time stamp arrays in Swift.
So, say I want to go from 0 to 4 seconds, I can use Array(0...4), which gives [0, 1, 2, 3, 4]
But how can I get [0.0, 0.5 1.0, 2.0, 2.5, 3.0, 3.5, 4.0]?
Essentially I want a flexible delta, such as 0.5, 0.05, etc.
You can use stride(from:through:by:):
let a = Array(stride(from: 0.0, through: 4.0, by: 0.5))
An alternative for non-constant increments (even more viable in Swift 3.1)
The stride(from:through:by:) functions as covered in #Alexander's answer is the fit for purpose solution where, but for the case where readers of this Q&A wants to construct a sequence (/collection) of non-constant increments (in which case the linear-sequence constructing stride(...) falls short), I'll also include another alternative.
For such scenarios, the sequence(first:next:) is a good method of choice; used to construct a lazy sequence that can be repeatedly queried for the next element.
E.g., constructing the first 5 ticks for a log10 scale (Double array)
let log10Seq = sequence(first: 1.0, next: { 10*$0 })
let arr = Array(log10Seq.prefix(5)) // [1.0, 10.0, 100.0, 1000.0, 10000.0]
Swift 3.1 is intended to be released in the spring of 2017, and with this (among lots of other things) comes the implementation of the following accepted Swift evolution proposal:
SE-0045: Add prefix(while:) and drop(while:) to the stdlib
prefix(while:) in combination with sequence(first:next) provides a neat tool for generating sequences with everything for simple next methods (such as imitating the simple behaviour of stride(...)) to more advanced ones. The stride(...) example of this question is a good minimal (very simple) example of such usage:
/* this we can do already in Swift 3.0 */
let delta = 0.05
let seq = sequence(first: 0.0, next: { $0 + delta})
/* 'prefix(while:)' soon available in Swift 3.1 */
let arr = Array(seq.prefix(while: { $0 <= 4.0 }))
// [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0]
// ...
for elem in sequence(first: 0.0, next: { $0 + delta})
.prefix(while: { $0 <= 4.0 }) {
// ...
}
Again, not in contest with stride(...) in the simple case of this Q, but very viable as soon as the useful but simple applications of stride(...) falls short, e.g. for a constructing non-linear sequences.
How can one create an array filled with values within a range (having a begin and end value) and a step? It should support begin and end values of float type.
For floats with custom stepping you can use Numeric#step like so:
-1.25.step(by: 0.5, to: 1.25).to_a
# => [-1.25, -0.75, -0.25, 0.25, 0.75, 1.25]
If you are looking on how to do this with integer values only, see this post or that post on how to create ranges and simply call .to_a at the end. Example:
(-1..1).step(0.5).to_a
# => [-1.0, -0.5, 0.0, 0.5, 1.0]