I have a panel data set (the identifiers are the variables of cc and year).
I have all the values for the Y variable but I have only the initial (X_{1996}) values of the X variable.
I would like to replace the missing values of X with this pattern:
X_{1997} = Y_{1996} + 0.95*X_{1996}
X_{1998} = Y_{1997} + 0.95*X_{1997}
.
.
.
X_{2015} = Y_{2014} + 0.95*X_{2014}
I would like to iteratively replace X starting from 1997, that is using the last generated value.
Is it possible to write a loop for this purpose?
Example data:
clear
input str3 cc year X Y
"ABW" 1996 467.5953 28.46004
"ABW" 1997 . 30.23142
"ABW" 1998 . 31.70577
"ABW" 1999 . 30.870903
"ABW" 2000 . 25.495903
"ABW" 2001 . 23.71254
"ABW" 2002 . 26.906475
"ABW" 2003 . 29.215036
"ABW" 2004 . 28.77914
"ABW" 2005 . 33.581017
"ABW" 2006 . 35.16129
"ABW" 2007 . 32.51442
"ABW" 2008 . 33.251934
"ABW" 2009 . 30.29287
"ABW" 2010 . 28.931993
"ABW" 2011 . 28.5057
"ABW" 2012 . 26.647564
"ABW" 2013 . 23.999134
"ABW" 2014 . 22.917984
"ABW" 2015 . 21.481943
"AFG" 1996 362.353 11.51442
"AFG" 1997 . 11.557285
"AFG" 1998 . 12.19376
"AFG" 1999 . 14.557285
"AFG" 2000 . 12.906254
"AFG" 2001 . 13.156855
"AFG" 2002 . 11.557285
"AFG" 2003 . 15.758896
"AFG" 2004 . 19.005346
"AFG" 2005 . 23.193766
"AFG" 2006 . 26.407976
"AFG" 2007 . 20.068926
"AFG" 2008 . 19.021326
"AFG" 2009 . 17.991436
"AFG" 2010 . 17.959072
"AFG" 2011 . 16.674591
"AFG" 2012 . 16.906254
"AFG" 2013 . 17.343513
"AFG" 2014 . 18.156855
"AFG" 2015 . 19.454687
If you want to calculate the missing values of X by using the last generated observation, then you can do so as follows:
bysort cc (year): replace X = Y[_n-1] + 0.95 * X[_n-1] if _n > 1
list, sepby(cc)
+----------------------------------+
| cc year X Y |
|----------------------------------|
1. | ABW 1996 467.5953 28.46004 |
2. | ABW 1997 472.6756 30.23142 |
3. | ABW 1998 479.2732 31.70577 |
4. | ABW 1999 487.0153 30.8709 |
5. | ABW 2000 493.5355 25.4959 |
6. | ABW 2001 494.3546 23.71254 |
7. | ABW 2002 493.3494 26.90648 |
8. | ABW 2003 495.5884 29.21504 |
9. | ABW 2004 500.024 28.77914 |
10. | ABW 2005 503.802 33.58102 |
11. | ABW 2006 512.1929 35.16129 |
12. | ABW 2007 521.7445 32.51442 |
13. | ABW 2008 528.1717 33.25193 |
14. | ABW 2009 535.015 30.29287 |
15. | ABW 2010 538.5571 28.93199 |
16. | ABW 2011 540.5613 28.5057 |
17. | ABW 2012 542.0389 26.64756 |
18. | ABW 2013 541.5845 23.99913 |
19. | ABW 2014 538.5045 22.91798 |
20. | ABW 2015 534.4972 21.48194 |
|----------------------------------|
21. | AFG 1996 362.353 11.51442 |
22. | AFG 1997 355.7498 11.55729 |
23. | AFG 1998 349.5196 12.19376 |
24. | AFG 1999 344.2373 14.55729 |
25. | AFG 2000 341.5828 12.90625 |
26. | AFG 2001 337.4099 13.15685 |
27. | AFG 2002 333.6962 11.55729 |
28. | AFG 2003 328.5687 15.7589 |
29. | AFG 2004 327.8992 19.00535 |
30. | AFG 2005 330.5096 23.19377 |
31. | AFG 2006 337.1778 26.40798 |
32. | AFG 2007 346.7269 20.06893 |
33. | AFG 2008 349.4595 19.02133 |
34. | AFG 2009 351.0078 17.99144 |
35. | AFG 2010 351.4489 17.95907 |
36. | AFG 2011 351.8355 16.67459 |
37. | AFG 2012 350.9183 16.90625 |
38. | AFG 2013 350.2787 17.34351 |
39. | AFG 2014 350.1082 18.15685 |
40. | AFG 2015 350.7597 19.45469 |
+----------------------------------+
Related
Note: this is about the same situation as How to download files with GET and preserve directory structure? One of these might be an X-Y question. I also asked on the Snowflake Community forum: https://community.snowflake.com/s/question/0D53r0000BZbU70CQF/prevent-filename-conflicts-when-unloading-files-with-partition-by.
I ran COPY INTO #mystage with PARTITION BY and SINGLE = false, creating several files in a stage with names like this:
+------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------+
| name | size | md5 | last_modified |
|------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------|
| mystage/00/data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_4_0.snappy.parquet | 8645856 | 2e3fe0e95b3b4d39f1b2c2cd2a5e5fd3 | Tue, 7 Jun 2022 13:51:40 GMT |
| mystage/01/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_1_0.snappy.parquet | 8743968 | 4d59bb8cfa355c8d55f05b88aa91002f | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/02/data_01a4c9ff-0602-1baa-0070-30030078eb32_1412_1_0.snappy.parquet | 9719360 | c723aae191fb46c1c83eb660e3dd41cc | Tue, 7 Jun 2022 13:51:43 GMT |
| mystage/03/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_2_0.snappy.parquet | 7786640 | b63ed8a15f18cd97ed8aaeb4fea89dee | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/04/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_1_0.snappy.parquet | 9249120 | 47471b0b0fb72b8d9ae5982176f29af1 | Tue, 7 Jun 2022 13:52:00 GMT |
| mystage/05/data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet | 9385936 | 18b286beb69b876aa3b4e08e857f69e7 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/06/data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet | 9187536 | 2dd0e99c7fa22fe42c624e4298c9d8f1 | Tue, 7 Jun 2022 13:51:58 GMT |
| mystage/07/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_4_0.snappy.parquet | 8804096 | aaf089af92251957502665fb7ab144f5 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/08/data_01a4c9ff-0602-1baa-0070-30030078eb32_312_5_0.snappy.parquet | 9214480 | 7969198b4e826d791e6f99455d01d72c | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/09/data_01a4c9ff-0602-1baa-0070-30030078eb32_512_5_0.snappy.parquet | 8986608 | 159c831c75439dcdc2743dabd91a66bb | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/10/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_7_0.snappy.parquet | 9063392 | 5410728fe0723f350322075f89a7e37f | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/11/data_01a4c9ff-0602-1baa-0070-30030078eb32_912_7_0.snappy.parquet | 9197600 | 68f6c4f81c3d3890a9eb269bfcde97ec | Tue, 7 Jun 2022 13:51:39 GMT |
| mystage/12/data_01a4c9ff-0602-1baa-0070-30030078eb32_1512_0_0.snappy.parquet | 9613440 | 124fdd720b104d4e6a119140409454a3 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/13/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_6_0.snappy.parquet | 8524336 | 17ace7467c50a89d6558a263b40f4036 | Tue, 7 Jun 2022 13:51:40 GMT |
| mystage/14/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_5_0.snappy.parquet | 8830192 | 83bf9a6e027e812583b69d735adda12c | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/15/data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_3_0.snappy.parquet | 9051568 | a75520c1713b6e4cc7a3f28ac4589f42 | Tue, 7 Jun 2022 13:51:40 GMT |
+------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------+
Note that there is some directory structure here. The directories 00, 01, etc. were created by the COPY INTO command.
As I described in my other question (How to download files with GET and preserve directory structure?), there doesn't seem to be a way to download these files in a way that preserves the directory structure. This is a problem because at least two of the files have identical "tail" names (data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet), so downloading them all to the same directory causes problems.
As a workaround for the issue described in my linked question, is there any way to change how these file names are created, such that the directory names 00, 01, etc. are converted to filename prefixes? That is, I would want 00_data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_4_0.snappy.parquet instead of 00/data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_4_0.snappy.parquet.
Note: this is about the same situation as Unload data to a stage with partitions "flattened" One of these might be an X-Y question. I also asked on the Snowflake Community forum: https://community.snowflake.com/s/question/0D53r0000BZbU70CQF/prevent-filename-conflicts-when-unloading-files-with-partition-by.
I have some files in a Snowflake stage called mystage:
+------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------+
| name | size | md5 | last_modified |
|------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------|
| mystage/00/data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_4_0.snappy.parquet | 8645856 | 2e3fe0e95b3b4d39f1b2c2cd2a5e5fd3 | Tue, 7 Jun 2022 13:51:40 GMT |
| mystage/01/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_1_0.snappy.parquet | 8743968 | 4d59bb8cfa355c8d55f05b88aa91002f | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/02/data_01a4c9ff-0602-1baa-0070-30030078eb32_1412_1_0.snappy.parquet | 9719360 | c723aae191fb46c1c83eb660e3dd41cc | Tue, 7 Jun 2022 13:51:43 GMT |
| mystage/03/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_2_0.snappy.parquet | 7786640 | b63ed8a15f18cd97ed8aaeb4fea89dee | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/04/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_1_0.snappy.parquet | 9249120 | 47471b0b0fb72b8d9ae5982176f29af1 | Tue, 7 Jun 2022 13:52:00 GMT |
| mystage/05/data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet | 9385936 | 18b286beb69b876aa3b4e08e857f69e7 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/06/data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet | 9187536 | 2dd0e99c7fa22fe42c624e4298c9d8f1 | Tue, 7 Jun 2022 13:51:58 GMT |
| mystage/07/data_01a4c9ff-0602-1baa-0070-30030078eb32_712_4_0.snappy.parquet | 8804096 | aaf089af92251957502665fb7ab144f5 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/08/data_01a4c9ff-0602-1baa-0070-30030078eb32_312_5_0.snappy.parquet | 9214480 | 7969198b4e826d791e6f99455d01d72c | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/09/data_01a4c9ff-0602-1baa-0070-30030078eb32_512_5_0.snappy.parquet | 8986608 | 159c831c75439dcdc2743dabd91a66bb | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/10/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_7_0.snappy.parquet | 9063392 | 5410728fe0723f350322075f89a7e37f | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/11/data_01a4c9ff-0602-1baa-0070-30030078eb32_912_7_0.snappy.parquet | 9197600 | 68f6c4f81c3d3890a9eb269bfcde97ec | Tue, 7 Jun 2022 13:51:39 GMT |
| mystage/12/data_01a4c9ff-0602-1baa-0070-30030078eb32_1512_0_0.snappy.parquet | 9613440 | 124fdd720b104d4e6a119140409454a3 | Tue, 7 Jun 2022 13:51:44 GMT |
| mystage/13/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_6_0.snappy.parquet | 8524336 | 17ace7467c50a89d6558a263b40f4036 | Tue, 7 Jun 2022 13:51:40 GMT |
| mystage/14/data_01a4c9ff-0602-1baa-0070-30030078eb32_112_5_0.snappy.parquet | 8830192 | 83bf9a6e027e812583b69d735adda12c | Tue, 7 Jun 2022 13:51:41 GMT |
| mystage/15/data_01a4c9ff-0602-1baa-0070-30030078eb32_1112_3_0.snappy.parquet | 9051568 | a75520c1713b6e4cc7a3f28ac4589f42 | Tue, 7 Jun 2022 13:51:40 GMT |
+------------------------------------------------------------------------------------------------------------+---------+----------------------------------+------------------------------+
I tried to download these files:
GET #mystage file:///data;
But the files were downloaded without the 00, 01, etc. directory structure. At least two of the files have non-unique names (in this case data_01a4c9ff-0602-1baa-0070-30030078eb32_612_2_0.snappy.parquet), so this means I can't actually download all of the files!
In this particular case, it causes a "file not found" error, but in general I want some way to preserve the directory structure when downloading. Is there some other Snowflake command I can use for this purpose?
I currently have deviations for a number of models as shown in the data below:
year model_2015 model_2016 model_2017
2016 15 . .
2017 20 10 .
2018 30 20 30
Variable Model_2015, performed in 2015, has deviations for 2016, 2017, 2018; variable Model_2016 for 2017 as well as 2018, and so on.
I would like to create a variable that sums the first observation of each.
So for this example:
first = 15 + 10 + 30 = 55
I'm assuming that I have to make a loop, but I am stumped on how to go about it.
EDIT:
Ideally, I would also like a solution adding the second, third, and so on non-missing observations.
The following works for me:
generate first = model_2015[1] + model_2016[2] + model_2017[3]
However, here is a more general approach:
clear
input year model_2015 model_2016 model_2017
2016 15 . .
2017 20 10 .
2018 30 20 30
end
generate id = 1
tempfile myfile
save `myfile'
collapse (firstnm) model*, by(id)
egen first = rowtotal(model*)
keep id first
merge 1:m id using `myfile'
drop id _merge
order year model* first
list, abbreviate(15)
+-----------------------------------------------------+
| year model_2015 model_2016 model_2017 first |
|-----------------------------------------------------|
1. | 2016 15 . . 55 |
2. | 2017 20 10 . 55 |
3. | 2018 30 20 30 55 |
+-----------------------------------------------------+
EDIT:
Below, is an even more general solution:
clear
input year model_2015 model_2016 model_2017
2016 15 . .
2017 20 10 .
2018 30 20 30
2019 40 10 10
end
local i = 0
foreach v of varlist model* {
local ++i
local vals
forvalues j = 1 / `=_N' {
if !missing(`v'[`j']) local vals `vals' `=`v'[`j']'
}
local ind_`i' `: word 1 of `vals'' // CHANGE THIS NUMBER
local ind_all `ind_all' `ind_`i''
}
generate first = `= subinstr("`ind_all'", " ", "+", `= wordcount("`ind_all'") - 1')'
Results:
list, abbreviate(15)
+-----------------------------------------------------+
| year model_2015 model_2016 model_2017 first |
|-----------------------------------------------------|
1. | 2016 15 . . 55 |
2. | 2017 20 10 . 55 |
3. | 2018 30 20 30 55 |
4. | 2019 40 10 10 55 |
+-----------------------------------------------------+
+-----------------------------------------------------+
| year model_2015 model_2016 model_2017 second |
|-----------------------------------------------------|
1. | 2016 15 . . 50 |
2. | 2017 20 10 . 50 |
3. | 2018 30 20 30 50 |
4. | 2019 40 10 10 50 |
+-----------------------------------------------------+
+-----------------------------------------------------+
| year model_2015 model_2016 model_2017 third |
|-----------------------------------------------------|
1. | 2016 15 . . 40 |
2. | 2017 20 10 . 40 |
3. | 2018 30 20 30 40 |
4. | 2019 40 10 10 40 |
+-----------------------------------------------------+
Note that in this case I used a slightly modified example for better illustration.
The code below might be the loop(s) that you are seeking for:
forvalues i = 1 / `=_N' {
generate S_`i' = 0
forvalues j = `i' / `=_N' {
capture replace S_`i' = S_`i' + model_`=2015+`j'-`i''[`j']
}
}
I haven't been able to find a function or query that would show me a PID that owns the connection.
these tables were created on the 20th and I haven't been able to find which open database connection owns it.
pg_temp_57.tmplb1604202242425
pg_temp_57.tmplb1604202239538
pg_temp_60.tmplb1604202242006
mainDB=# select relname,relcreatetime,relfrozenxid from pg_class where relname ~ 'tmplb16042022';
relname | relcreatetime | relfrozenxid
----------------------------------+---------------+--------------
tmplb1604202242425 | 1461206562 | 372689
tmplb1604202239538 | 1461206393 | 372487
tmplb1604202242006 | 1461206520 | 372626
(6 rows)
[root#postgreDB]# date -d #1461206562
Wed Apr 20 22:42:42 EDT 2016
[root#postgreDB]# date -d #1461206520
Wed Apr 20 22:42:00 EDT 2016
[root#postgreDB]# date -d #1461206393
Wed Apr 20 22:39:53 EDT 2016
[root#postgreDB]# date -d #1461206562
Wed Apr 20 22:42:42 EDT 2016
I'm having a problem with a function i'm working on. I am able to get the proper data, just not print out the names of the month as well.
sample output i'm getting:
| Month | High | Low | Avg | Precip |
|-----------|-------|-------|-------|---------|
1 | 9.8 | -26.2 | -7.8 | 55.3 |
2 | 7.5 | -23.3 | -8.6 | 33.1 |
3 | 14.2 | -19.6 | -4.7 | 33.2 |
4 | 23.7 | -5.3 | 6.2 | 56.8 |
5 | 33.0 | -0.6 | 13.9 | 62.7 |
6 | 32.1 | 8.0 | 19.7 | 69.7 |
7 | 34.9 | 12.6 | 22.2 | 181.8 |
8 | 31.5 | 11.0 | 20.9 | 69.2 |
9 | 34.1 | 5.0 | 16.1 | 69.0 |
10 | 24.8 | -2.9 | 10.8 | 56.9 |
11 | 16.0 | -12.8 | 2.1 | 36.2 |
12 | 15.6 | -17.8 | -4.2 | 65.8
I want to conver digits 1-12 to there proper month names. Ie: 1 = January.
void printMonthlyStatistic(int month,const struct MonthlyStatistic* monthly)
is what the function looks like
for(i=0;i<12;i++){
printMonthlyStatistic(i+1,&monthly[i])
and is called in main as such.
Any help onto where to look for the proper method to complete this would be great. Thanks!
You should use an array to store the name of the month, like:
const char * months[12] = {"January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"};
You can get the name of the month easily, for example, if index = 1, then
months[index -1]
will give you the first month name: January.
The following code:
for (int i = 0; i < 12; ++i) {
printf("%s ", months[i]);
}
will output:
January February March April May June July August September October
November December
You can maintain Month Array of String like
char months_names[12] = {Jan, Feb, Mar, Apr, Jun, Jul, Aug, Sep, Oct, Nov, Dec};
You can Just paas integer number month to array for getting String value
months_names["Integrer Value"] You will get String name of month.