I want to export a single date and variable fwi of a xarray dataset ds to a tif file. However, my dataarray has too many dimensions and I cannot figure out how to effectively remove the dimension lsm_time.
ds
<xarray.Dataset>
Dimensions: (time: 2436, y: 28, x: 58, lsm_time: 1)
Coordinates:
* time (time) datetime64[ns] 2015-05-02T12:00:00 ... 2021-12-31T12:...
step <U8 '12:00:00'
surface float64 0.0
* y (y) float64 36.01 36.11 36.21 36.31 ... 38.41 38.51 38.61 38.71
* x (x) float64 -7.64 -7.54 -7.44 -7.34 ... -2.24 -2.14 -2.04 -1.94
spatial_ref int32 0
Dimensions without coordinates: lsm_time
Data variables:
ffmc (time, y, x, lsm_time) float32 nan nan nan ... 88.93 88.53
dmc (time, y, x, lsm_time) float32 nan nan nan ... 6.511 7.908 8.45
dc (time, y, x, lsm_time) float32 nan nan nan ... 406.2 428.5
isi (time, y, x, lsm_time) float32 nan nan nan ... 3.872 3.852
bui (time, y, x, lsm_time) float32 nan nan nan ... 15.08 16.11
fwi (time, y, x, lsm_time) float32 nan nan nan ... 5.303 5.486
Exporting the dataarray raises the error TooManyDimensions:
ds.fwi.sel(time="07-14-2021").rio.to_raster("file_out.tif")
raise TooManyDimensions(
rioxarray.exceptions.TooManyDimensions: Only 2D and 3D data arrays supported. Data variable: fwi
I already dropped the dimension lsm_time in a previous step, when I masked the dataset ds with a land sea mask lsm (single date), and had to duplicate/add the time dimension of the dataset ds. So maybe this is an error resulting from this data handling.. However, I could figure out how to mask otherwise.
lsm
Dimensions: (x: 58, y: 28, time: 1)
Coordinates:
* x (x) float64 -7.64 -7.54 -7.44 -7.34 ... -2.24 -2.14 -2.04 -1.94
* y (y) float64 36.01 36.11 36.21 36.31 ... 38.41 38.51 38.61 38.71
* time (time) datetime64[ns] 2013-11-29
Data variables:
spatial_ref int32 0
lsm (time, y, x) float64 0.0 0.0 0.0 0.0 0.0 ... 1.0 1.0 1.0 0.9996
lsm = lsm.expand_dims({"new_time" : ds.time})
lsm = lsm.rename({"time":"lsm_time"}).rename({"new_time" : "time"}).drop("lsm_time") #issue here: drop_dims removes variable..
ds = ds.where(lsm>=threshold)
So I already applied .drop("lsm_time")
But, it is still
print(ds.fwi.sel(time="07-14-2021").dims)
> ('time', 'y', 'x', 'lsm_time')
When trying .drop_dims, it removes all data variables.
ds.drop_dims('lsm_time')
<xarray.Dataset>
Dimensions: (time: 2436, y: 28, x: 58)
Coordinates:
* time (time) datetime64[ns] 2015-05-02T12:00:00 ... 2021-12-31T12:...
step <U8 '12:00:00'
surface float64 0.0
* y (y) float64 36.01 36.11 36.21 36.31 ... 38.41 38.51 38.61 38.71
* x (x) float64 -7.64 -7.54 -7.44 -7.34 ... -2.24 -2.14 -2.04 -1.94
spatial_ref int32 0
Data variables:
*empty*
What am I missing or what did I do wrong? Thanks for any help!
Dimensions with size 1 can be removed using the .squeeze method.
Conversely, you can add a dimension of size 1 using .expand_dims
import xarray as xr
x = xr.tutorial.load_dataset("rasm")
y = x.isel(time=slice(0,1))
y.dims
# Frozen({'time': 1, 'y': 205, 'x': 275})
y.squeeze().dims
# Frozen({'y': 205, 'x': 275})
y.squeeze().expand_dims("newdim").dims
# Frozen({'y': 205, 'x': 275, 'newdim': 1})
Related
I want to fit an intensity distribution function to 2D image data using scipy.optimize.curve_fit and can't locate the error in my code:
# Define doughnut beam intensity distribution function
def doughnut(x, y, x0, y0, A, FWHM):
'''2D intensity distribution function of doughnut beams (DOI: 10.1126/science.aak9913,
https://science.sciencemag.org/content/sci/suppl/2016/12/21/science.aak9913.DC1/Balzarotti_SM.pdf).
Parameters
----------
x, y : float
X and Y coordinates, orthogonal to beam axis
x0 : float
X offset
y0 : float
Y offset
A : float
Peak intensity
FWHM : float
Full width at half maximum
'''
return A*np.exp(1)*4*np.log(2)*(np.dot(x+x0,x+x0) + np.dot(y+y0,y+y0))/FWHM**2*np.exp(-4*np.log(2)*(np.dot(x+x0,x+x0) + np.dot(y+y0,y+y0))/FWHM**2)
# Read image file names
pathname = '/home/user/doughnut_beam/'
filenameList = [filename for filename in os.listdir(pathname)
if filename.endswith('.tif')]
# Open image files, fit doughnut beam intensity distribution function
for filename in filenameList:
img = Image.open(pathname + filename)
X, Y = img.size
xRange = np.arange(1, X+1)
yRange = np.arange(1, Y+1)
xGrid, yGrid = np.meshgrid(xRange, yRange)
xyGrid = np.vstack((xGrid.ravel(), yGrid.ravel())) # scipy.optimize.curve_fit requires 2xN-array
imgArray = np.array(img)
imgArrayFlat = imgArray.ravel() # Flatten 2D pixel data into 1D array for scipy.optimize.curve_fit
params_opt, params_cov = curve_fit(doughnut, xyGrid, imgArrayFlat)
This is the output from Jupyter Notebook:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-44-eaa3ebdb6469> in <module>()
17 imgArrayFlat = imgArray.ravel() # Flatten 2D pixel data into 1D array for scipy.optimize.curve_fit
18
---> 19 params_opt, params_cov = curve_fit(doughnut, xyGrid, imgArrayFlat)
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in curve_fit(f, xdata, ydata, p0, sigma, absolute_sigma, check_finite, bounds, method, jac, **kwargs)
749 # Remove full_output from kwargs, otherwise we're passing it in twice.
750 return_full = kwargs.pop('full_output', False)
--> 751 res = leastsq(func, p0, Dfun=jac, full_output=1, **kwargs)
752 popt, pcov, infodict, errmsg, ier = res
753 cost = np.sum(infodict['fvec'] ** 2)
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
381 if not isinstance(args, tuple):
382 args = (args,)
--> 383 shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
384 m = shape[0]
385 if n > m:
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in _check_func(checker, argname, thefunc, x0, args, numinputs, output_shape)
25 def _check_func(checker, argname, thefunc, x0, args, numinputs,
26 output_shape=None):
---> 27 res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
28 if (output_shape is not None) and (shape(res) != output_shape):
29 if (output_shape[0] != 1):
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in func_wrapped(params)
461 if transform is None:
462 def func_wrapped(params):
--> 463 return func(xdata, *params) - ydata
464 elif transform.ndim == 1:
465 def func_wrapped(params):
<ipython-input-43-3e0adae6fbe0> in doughnut(x, y, x0, y0, A, FWHM)
17 Full width at half maximum
18 '''
---> 19 return A*np.exp(1)*4*np.log(2)*(np.dot(x+x0,x+x0) + np.dot(y+y0,y+y0))/FWHM**2*np.exp(-4*np.log(2)*(np.dot(x+x0,x+x0) + np.dot(y+y0,y+y0))/FWHM**2)
ValueError: shapes (2,210) and (2,210) not aligned: 210 (dim 1) != 2 (dim 0)
UPDATE: For some reason, using numpy.dot to square the (offset) variables x+x0 and y+y0 in the function definition does not work. Simply changing to the ** operator results in the correct plot:
# UPDATED: Define doughnut beam intensity distribution function
def doughnut(x, y, x0, y0, A, FWHM):
'''2D intensity distribution function of doughnut beams (DOI: 10.1126/science.aak9913,
https://science.sciencemag.org/content/sci/suppl/2016/12/21/science.aak9913.DC1/Balzarotti_SM.pdf).
Parameters
----------
x, y : float
X and Y coordinates, orthogonal to beam axis
x0 : float
X offset
y0 : float
Y offset
A : float
Peak intensity
FWHM : float
Full width at half maximum
'''
return A*np.exp(1)*4*np.log(2)*((x+x0)**2 + (y+y0)**2)/FWHM**2*np.exp(-4*np.log(2)*((x+x0)**2 + (y+y0)**2)/FWHM**2)
fig = plt.figure()
ax = fig.gca(projection='3d')
# Make data
X = np.arange(-10, 10, 0.25)
Y = np.arange(-10, 10, 0.25)
X, Y = np.meshgrid(X, Y)
Z = doughnut(X, Y, x0=0, y0=0, A=1.5, FWHM=7)
# Plot the surface
surf = ax.plot_surface(X, Y, Z)
plt.show()
=> Plot
BUT: Now I'm getting a new error when trying to fit the data:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-61-eaa3ebdb6469> in <module>()
17 imgArrayFlat = imgArray.ravel() # Flatten 2D pixel data into 1D array for scipy.optimize.curve_fit
18
---> 19 params_opt, params_cov = curve_fit(doughnut, xyGrid, imgArrayFlat)
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in curve_fit(f, xdata, ydata, p0, sigma, absolute_sigma, check_finite, bounds, method, jac, **kwargs)
749 # Remove full_output from kwargs, otherwise we're passing it in twice.
750 return_full = kwargs.pop('full_output', False)
--> 751 res = leastsq(func, p0, Dfun=jac, full_output=1, **kwargs)
752 popt, pcov, infodict, errmsg, ier = res
753 cost = np.sum(infodict['fvec'] ** 2)
/usr/lib/python3/dist-packages/scipy/optimize/minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
384 m = shape[0]
385 if n > m:
--> 386 raise TypeError('Improper input: N=%s must not exceed M=%s' % (n, m))
387 if epsfcn is None:
388 epsfcn = finfo(dtype).eps
TypeError: Improper input: N=5 must not exceed M=2
This should do the trick. Take a look at the for_fitting function to see how you can package everything in a way curve_fit will accept.
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit
def doughnut(y, x, y0, x0, A, FWHM):
"""2D intensity distribution function of doughnut beams (DOI: 10.1126/science.aak9913,
https://science.sciencemag.org/content/sci/suppl/2016/12/21/science.aak9913.DC1/Balzarotti_SM.pdf).
Parameters
----------
y, x : float
X and Y coordinates, orthogonal to beam axis
y0 : float
Y offset
x0 : float
X offset
A : float
Peak intensity
FWHM : float
Full width at half maximum
"""
return (
A
* np.e
* 4
* np.log(2)
* ((x + x0) ** 2 + (y + y0) ** 2)
/ FWHM ** 2
* np.exp(-4 * np.log(2) * ((x + x0) ** 2 + (y + y0) ** 2) / FWHM ** 2)
)
fig0, (ax0, ax1, ax2) = plt.subplots(1, 3, sharex=True, sharey=True)
# Make data
X = np.arange(-10, 10, 0.25)
Y = np.arange(-10, 10, 0.25)
X, Y = np.meshgrid(X, Y)
true_params = (0, 0, 100, 7)
Z = doughnut(Y, X, *true_params)
# Plot the surface
ax0.matshow(Z, extent=(-10, 10, 10, -10))
ax0.set_title("Ground Truth")
def for_fitting(xdata, y0, x0, A, FWHM):
yy, xx = xdata
return doughnut(yy, xx, y0, x0, A, FWHM).ravel()
noisy_data = np.random.poisson(Z) + np.random.randn(*Z.shape)
ax1.matshow(noisy_data, extent=(-10, 10, 10, -10))
ax1.set_title("Noisy Data")
opt_params, cov = curve_fit(for_fitting, (Y, X), noisy_data.ravel(), p0=(0, 0, 10, 1))
print(opt_params)
fit_Z = doughnut(Y, X, *opt_params)
ax2.matshow(fit_Z, extent=(-10, 10, 10, -10))
ax2.set_title("Fit")
fig0.tight_layout()
plt.show()
I have 2 lists of x and y coordinate that are independently generated, with a/h amount of points between 0 and a.
x = np.linspace(0, a, a/h)
y = np.linspace(0, d, d/h)
when a/h is such that 0 increases to a in steps of integers i.e. [0,1,2,..,a]. It's nice because then the number of elements within the list can be used as indices. And as a result I can usually create a meshgrid such that a third list V1 can be associated with it.
X, Y = plt.meshgrid(x, y)
def potential(V1):
return V1[X, Y]
where potential(V1) is now V1 corresponding to the meshgrid [x, y]. However I'm doing an assignment where I'm required to investigate how step-sizes affect my problem. As a result if I was to have a step-size of non-integers from 0 to a i.e. [0, 0.5, 1,...,a] Now I can't do what I did above since the indices are now non-integers. Raising the error
IndexError: arrays used as indices must be of integer (or boolean) type
How can I fix this so that I don't rely on the value of the element itself as the index of the elements, so that if there was a step-size of 0.25 between 0 to a for a list X say i.e.
X = [0, 0.25, 0.75. 1.0] or x = np.linspace(0,1,4)
such that I can have
x[0] = 0 corresponds to V[0]
x[1] = 0.25 corresponds to V[1]
x[2] = 0.75 corresponds to V[2]
x[3] = 1 corresponds to V[3]
?
How to get meter of STDistance() or STLength()?
I use
linestring(x y z, x y z, x y z, ...).STLength() //
point(x y).STDistance(point(x y))
but object has 0 of SRID...can I get meter length?
I've been fumbling with this for a while now but let me start with the beginning.
This is how the first few lines of my data look like:
Year Action_mean_global Adventure_mean_global Fighting_mean_global Misc_mean_global Platform_mean_global
1 1980 0.3400000 NaN 0.77 0.6775 NaN
2 1981 0.5936000 NaN NaN NaN 2.3100
3 1982 0.3622222 NaN NaN 0.8700 1.0060
4 1983 0.4085714 0.4 NaN 2.1400 1.3860
5 1984 1.8500000 NaN NaN 1.4500 0.6900
6 1985 1.7600000 NaN 1.05 NaN 10.7925
Puzzle_mean_global Racing_mean_global Roleplaying_mean_global Shooter_mean_global Simulation_mean_global
1 NaN NaN NaN 3.53500 NaN
2 1.120000 0.480000 NaN 1.00400 0.45
3 3.343333 0.785000 NaN 0.75800 NaN
4 0.780000 NaN NaN 0.48000 NaN
5 1.046667 1.983333 NaN 10.36667 NaN
6 0.802500 NaN NaN 1.00000 0.03
Sports_mean_global Strategy_mean_global Total_mean_global
1 0.4900 NaN 1.2644444
2 0.1975 NaN 0.7776087
3 0.5250 NaN 0.8016667
4 3.2000 NaN 0.9876471
5 3.0900 NaN 3.5971429
6 1.9600 NaN 3.8528571
They are all numeric.
Now, I simply wanted to do a plot with ggplot() + geom_line() to visualize change over year per genre. It works when doing it step by step, i.e.:
ggplot(df)+
geom_line(aes_string(x = 'Year', y = plot_vector[1]))
geom_line(aes_string(x = 'Year', y = plot_vector[2]))+
geom_line(aes_string(x = 'Year', y = plot_vector[3]))+
geom_line(aes_string(x = 'Year', y = plot_vector[4]))+
geom_line(aes_string(x = 'Year', y = plot_vector[5]))+
geom_line(aes_string(x = 'Year', y = plot_vector[6]))+
geom_line(aes_string(x = 'Year', y = plot_vector[7]))+
geom_line(aes_string(x = 'Year', y = plot_vector[8]))+
geom_line(aes_string(x = 'Year', y = plot_vector[9]))+
geom_line(aes_string(x = 'Year', y = plot_vector[10]))+
geom_line(aes_string(x = 'Year', y = plot_vector[11]))+
geom_line(aes_string(x = 'Year', y = plot_vector[12]))
(plot_vector simply contains all column-names except for Year)
However, doing it in a for-loop:
p1 <- ggplot(df)+
geom_line(aes_string(x = 'Year', y = plot_vector[1]))
for (plotnumber in 2:length(plot_vector))
{
p1 <- p1 +
geom_line(aes_string(x = 'Year', y = plot_vector[plotnumber]))
}
I get the error message. Anyone can muster an idea?
Adding lines with a for loop to a ggplot object may have caused the reported error message but has a general problem caused by lazy evaluation. This has been asked frequently on SO, see, e.g., “for” loop only adds the final ggplot layer, or ggplot loop adding curves fails, but works one at a time.
However, ggplot2 works best when data are supplied in long format. Here, melt()from the data.table package is used to reshape df:
library(data.table)
molten <- melt(setDT(df), id.vars = c("Year"))
library(ggplot2)
ggplot(molten, aes(x = Year, y = value, group = variable, colour = variable)) +
geom_line()
This creates the following chart:
Data
df <- structure(list(Year = 1980:1985, Action_mean_global = c(0.34,
0.5936, 0.3622222, 0.4085714, 1.85, 1.76), Adventure_mean_global = c(NaN,
NaN, NaN, 0.4, NaN, NaN), Fighting_mean_global = c(0.77, NaN,
NaN, NaN, NaN, 1.05), Misc_mean_global = c(0.6775, NaN, 0.87,
2.14, 1.45, NaN), Platform_mean_global = c(NaN, 2.31, 1.006,
1.386, 0.69, 10.7925), Puzzle_mean_global = c(NaN, 1.12, 3.343333,
0.78, 1.046667, 0.8025), Racing_mean_global = c(NaN, 0.48, 0.785,
NaN, 1.983333, NaN), Roleplaying_mean_global = c(NaN, NaN, NaN,
NaN, NaN, NaN), Shooter_mean_global = c(3.535, 1.004, 0.758,
0.48, 10.36667, 1), Simulation_mean_global = c(NaN, 0.45, NaN,
NaN, NaN, 0.03), Sports_mean_global = c(0.49, 0.1975, 0.525,
3.2, 3.09, 1.96), Strategy_mean_global = c(NaN, NaN, NaN, NaN,
NaN, NaN), Total_mean_global = c(1.2644444, 0.7776087, 0.8016667,
0.9876471, 3.5971429, 3.8528571)), .Names = c("Year", "Action_mean_global",
"Adventure_mean_global", "Fighting_mean_global", "Misc_mean_global",
"Platform_mean_global", "Puzzle_mean_global", "Racing_mean_global",
"Roleplaying_mean_global", "Shooter_mean_global", "Simulation_mean_global",
"Sports_mean_global", "Strategy_mean_global", "Total_mean_global"
), class = "data.frame", row.names = c(NA, -6L))
Given the 2x2 unitary matrix representation of an operation to apply to a single qubit, how do I figure out the rotation it corresponds to on the Bloch sphere?
For example, the Hadamard matrix is a 180 degree rotation around the X+Z axis. How do I get from [[1,1],[1,-1]]*sqrt(0.5) to (X+Z, 180 deg)?
Single-qubit operations are basically just unit quaternions, but with an extra phase factor. The similarity is because the Pauli matrices, times sqrt(-1), satisfy the i^2=j^2=k^2=ijk=-1 relation that defines quaternions.
As a result, the hard part of the conversion method is already taken care of by any "quaternion to axis angle" code. Just pull out the phased quaternion components, figure out the phase factor, then apply the quaternion-to-angle-axis method.
import math
import cmath
def toBlochAngleAxis(matrix):
"""
Breaksdown a matrix U into axis, angle, and phase_angle components satisfying
U = exp(i phase_angle) (I cos(angle/2) - axis sigma i sin(angle/2))
:param matrix: The 2x2 unitary matrix U
:return: The breakdown (axis(x, y, z), angle, phase_angle)
"""
[[a, b], [c, d]] = matrix
# --- Part 1: convert to a quaternion ---
# Phased components of quaternion.
wp = (a + d) / 2.0
xp = (b + c) / 2.0j
yp = (b - c) / 2.0
zp = (a - d) / 2.0j
# Arbitrarily use largest value to determine the global phase factor.
phase = max([wp, xp, yp, zp], key=abs)
phase /= abs(phase)
# Cancel global phase factor, recovering quaternion components.
w = complex(wp / phase).real
x = complex(xp / phase).real
y = complex(yp / phase).real
z = complex(zp / phase).real
# --- Part 2: convert from quaternion to angle-axis ---
# Floating point error may have pushed w outside of [-1, +1]. Fix that.
w = min(max(w, -1), +1)
# Recover angle.
angle = -2*math.acos(w)
# Normalize axis.
n = math.sqrt(x*x + y*y + z*z);
if n < 0.000001:
# There's an axis singularity near angle=0.
# Just default to no rotation around the Z axis in this case.
angle = 0
x = 0
y = 0
z = 1
n = 1
x /= n
y /= n
z /= n
# --- Part 3: (optional) canonicalize ---
# Prefer angle in [-pi, pi]
if angle <= -math.pi:
angle += 2*math.pi
phase *= -1
# Prefer axes that point positive-ward.
if x + y + z < 0:
x *= -1
y *= -1
z *= -1
angle *= -1
phase_angle = cmath.polar(phase)[1]
return (x, y, z), angle, phase_angle
Testing it out:
print(toBlochAngleAxis([[1, 0], [0, 1]])) # Identity
# ([0, 0, 1], 0, 0.0)
print(toBlochAngleAxis([[0, 1], [1, 0]])) # Pauli X, 180 deg around X
# ([1.0, -0.0, -0.0], 3.141592653589793, 1.5707963267948966)
print(toBlochAngleAxis([[0, -1j], [1j, 0]])) # Pauli Y, 180 deg around Y
# ([-0.0, 1.0, -0.0], 3.141592653589793, 1.5707963267948966)
print(toBlochAngleAxis([[1, 0], [0, -1]])) # Pauli Z, 180 deg around Z
# ([-0.0, -0.0, 1.0], 3.141592653589793, 1.5707963267948966)
s = math.sqrt(0.5)
print(toBlochAngleAxis([[s, s], [s, -s]])) # Hadamard, 180 deg around X+Z
# ([0.7071067811865476, -0.0, 0.7071067811865476], 3.141592653589793, 1.5707963267948966)
print(toBlochAngleAxis([[s, s*1j], [s*1j, s]])) # -90 deg X axis, no phase
# ((1.0, 0.0, 0.0), -1.5707963267948966, 0.0)