AS400 cpyf not copying a character record - file

I am trying to copy all of the records from a data file (STUDMARKS) into my physical file (MARKS) using the CPYF command.
A R MARKSR TEXT('Marks Records')
A STUDENTID 9S 0 COLHDG('Student' 'ID')
A COURSE_CD 6A COLHDG('Course' 'Code')
A FINAL_MARK 3S COLHDG('Final' 'Mark')
A DATERUN L COLHDG('Date' 'Run')
A K STUDENTID
A K COURSE_CD
This is what I currently have in my MARKS.pf. The STUDMARKS.pf-dta file has the first three records already defined, the DATERUN record get filled with the date of use.
CPYF FROMFILE(IBC233LIB/STUDMARKS) TOFILE(DS233B32/MARKS) MBROPT(*REPLACE) FMTOPT(*MAP *DROP)
The above is the CPYF command that I ran after creating MARKS.pf, and after doing a RUNQRY to see all the records I've noticed that all but the COURSE_CD have been filled. COURSE_CD is completely blank.
I did some research before hand and did a DSPFFD on both members to ensure that the record lengths and types were all the same, which they were. I did notice, however, that in STUDMARKS.pf-dta that all the records had a buffer length which was equivalent to the field length. The STUDENTID field in MARKS.pf was the only one to not share this property, where the field length is 9, but the buffer length is only 5. I'm not sure if it's the reason why I'm having such difficulty, and the matter is almost certainly less so than what I'm making it out to be, but I've been at this for quite some time and a just can't seem to copy records from one member to another.
It's incredibly frustrating, and help would be greatly appreciated
I took screen shots of the DSPFFD commands for both files
For STUDMARKS
And For MARKS
EDIT
Just now seeing the spelling error! Smashing my head against the desk but I almost guarantee that is the problem. All of your answers were very informative and helpful though, so thank you very much
EDITEDIT
for others, despite the fact that I did change the names when recompiling the program, it will not work unless you delete the file first and THEN compile it. Very frustrating, but that's just how it is...
So DLTF [file name] and then recompile

As James noted, the differences in buffer length for STUDENTID are due to one file having it defined as packed and the other having it defined as zoned.
This won't matter to CPYF as both are a compatible numeric and CPYF will map between them as you've seen.
However, this proves that there's more than just a missing field different between the two files. Use DSPFFD and look at post the definitions of COURSE_ID from both files.
I'd bet either the names are different or the types are.

What you are experiencing is the difference between a packed and a signed decimal field.
More than likely you forgot to specify a datatype in position 35 of the DDS specification for the STUDENTID field in the MARKS file.
For example:
A STUDENTID 9S 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID ZONED 9 0 9 1 Both Student
ID
A STUDENTID 9 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID ZONED 9 0 5 1 Both Student
ID
A STUDENTID 9P 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID PACKED 9 0 5 1 Both Student
ID
The explanation for this behaviour can be found in the DDS reference in the section Data type for physical and logical files (position 35):
For physical files, if you do not specify a data type or duplicate one from a referenced field, the operating system assigns the following defaults:
A (character) if the decimal positions 36 through 37 are blank.
P (packed decimal) if the decimal positions 36 through 37 contain a number in the range 0 through 63.
Because the data types are different the FMTOPT(*MAP *DROP) tells the CPYF command to silenty drop and default any non-matching fields.
The odd thing is the file field description identifies the field as ZONED when it is really PACKED.

The *DROP value for the FMTOPT parameter excludes like named fields that do not have the same attribute and relative position in both files. The COURSE_CD field has a different position in the receiving file.

Related

What line changed in a PDSE member?

Earlier today, I changed a source-member in a PDSE in z/OS 2.2. I can see from Data Set List Utility that it was changed by me today.
Name Prompt Size Created Changed ID
MYPROGRM 50 2021/09/08 2022/07/26 11:44:03 MYUSRID
When browsing the source-member, I can see that some of the lines have a date on the far-right side of the screen, but they don't appear to be consistent. If I manually change a line, it doesn't always update that date.
RUNSTATS TABLESPACE XXXX.YYYYYY
TABLE(ALL) INDEX(ALL) 00011004
SHRLEVEL REFERENCE
Is there any way to tell what lines of my PDSE member where changed (for instance) today?
Thank you for your help!
Dave
You might be able to see what lines were changed or added but not deleted. If you have 'STATS ON' and sequence numbers on for the member you are editing, then the sequence numbers on the far right of the screen correspond to the MM values for the member:
VV.MM Version number and modification level. The version number is set
to 1 and the modification level is set to 0 when the member is
created. The modification level is the number of times this version
has been modified. For example, 02.15 means version 2, modification
15. If a member name is just an alternate name for another member, ALIAS appears in this field.
https://www.ibm.com/docs/en/zos/2.2.0?topic=statistics-member-list-display-panel-fields
The standard sequence field is the last 8 characters for fixed-length
records, or the first 8 characters for variable-length records,
regardless of the programming language. Use NUMBER ON STD to generate
sequence numbers in the standard sequence field. For members of
partitioned data sets, the format of standard sequence numbers depends
on whether statistics are being generated. If statistics are being
generated, standard sequence numbers are 6 digits followed by a
2-digit modification level number. The level number flag reflects the
modification level of the member when the line was created or last
changed. If, for example, a sequence number field contains 00040002,
the line was added or last changed at modification level 02. The
sequence number is 000400.
If STATS mode is off, or if you are editing a sequential data set,
standard sequence numbers are 8 digits, right-justified within the
field.
https://www.ibm.com/docs/en/zos/2.1.0?topic=numbers-sequence-number-format-modification-level
So if this is true for the member you are editing, and you want to know what was a changed in the last edit session, then check the Modification level of the member in question in the member display list (you may have to scroll left or right):
Name Prompt Size Init Mod VV MM ID
_________ TEST *Edited 8 7 2 01.02 JOCS065
**End**
aadn you can see the 00, 01, 02 etc. MM levels in columns 79-80
****** ***************************** Top of Data ******************************
000100 my data 00010000
000200 my data 00020000
000300 my data 00030000
000400 my data 00040000
000500 my data 00050000
000510 more data 00051001
000600 my changed data 00060002
000700 my data 00070000
****** **************************** Bottom of Data ****************************
so to see only what was last changed (useful in a large dataset), get the MM level (e.g. 45) end then, when editing, run the commands x all;f '45' 79 all
Those characters in positions 72 - 80 are not necessarily dates. They could be, but if they are it's because someone manually made that true. ISPF Edit does not enforce that.
It's more likely that at some point someone turned NUMBER ON on the member, or that some of the lines were copied from another member that at some point had NUMBER ON turned on.
To your question, it is common for HSM backups to be made of datasets when they are changed. You may be able to recover a recent backup of your PDSE to a different name and then use the compare ISPF Edit primary command to compare the two versions and determine what changed.
There is no way to tell which lines were changed by looking at the file. The number in far-right of the screen is likely a sequence number. If other lines in your file do not have similar numeric values it is likely that your profile is set to NUMEBR OFF.
To see your profile type prof command on the ISPF editor command line, it should display information similar to this:
> Command ===> prof Scroll ===> CSR
****** ***************************** Top of Data ******************************
=PROF> ....C (FIXED - 80)....RECOVERY ON....NUMBER ON STD......................
=PROF> ....CAPS OFF....HEX OFF....NULLS ON STD....TABS OFF.....................
=PROF> ....AUTOSAVE ON....AUTONUM ON....AUTOLIST OFF....STATS ON...............
=PROF> ....PROFILE UNLOCK....IMACRO NONE....PACK OFF....NOTE ON................
=PROF> ....HILITE C LOGIC PAREN CURSOR FIND MARGINS(1,72)......................
That will tell you if sequence numbers are ON or OFF. If you want to get rid of sequence numbers you can use do the following sequence of commands: num followed by unnum command, otherwse if you want sequence nymbers use just the num command.

How to remove a specific string from a character on multiple rows of data?

I have a table with a column for ID, it is a varchar, this column however has to be 14 characters long. The vendor that supplies this table left some with 15 characters. The format of the number is 'FL000000123456.' So, we asked them and they told us to just remove off the leading zero. I need to run an update on this table to remove that first zero, but I do not how to do that. I ran the LEN function to bring me back the ones that have a LEN of > 14. I can replace or run an update on the individual basis, but the table has over 1 million records and about 45,000 of them have the > 14 error. I do not work on SQL most of my time, so I am not as adept as some of you will be, but any help I can get would be greatly appreciated.
output: FL0000000123456 -- 15 chars
desired output: FL000000123456 -- 14 chars
Assuming that therefore you want to remove the 3rd character, you could use STUFF:
SELECT STUFF(YourColumn,3,1,'') AS NewColumn
FROM dbo.YourTable
WHERE LEN(YourColumn) = 15;
Of course, this does mean if you have a value like 'FL1234567891234' you would end up with 'FL234567891234', but I am assuming that won't happen.

How to read a flat file and load it into 2 different SQL tables(different table structure) using SSIS 2019

I have a flat file which doesn't have the header record. The data except the trailing record is like a fixed width flat file with no delimiters.
Data in flat file looks like:
TOM ROLLS
DAVECHILLS
TOTAL2XYZ
Fixed Width data(first 2 lines as shown in the above flat file data)
ColumnName Start position End position
Name 1 4
Last_name 5 9
I want to load the data(till trailing record) in data_table and the trailing record(starting with Total) in another table. The data in the total table should look like
c1 c2
2 XYZ
For the data table, I am currently using "fixed width" and dividing the data into different column and it is working fine. Can you please help to load this last trailing record in a different table(Total table as discussed above)
You have not provided enough data for me to test because I can find several methods to load one row and accomplish what you are asking but those methods not necessarily work with multiple rows depending on the structure of you source data.
On the surface it appears that you simply need to make another flat file connection and define the starting and end position to extract only the data for the second table.

Remove duplicate values based on timestamp

I would need your help with and SQL query that has to remove duplicate entries from a table, mostly using the datestamp column as a criteria in two passes.
Microsoft SQL DBMS is in question.
Here is a little more details:
Terminology: Module is basically a group of single machine workplaces onto which users operate.
Table:
ModNam column is fixed, there are 15 modules from M A01 to M A15, then goes the B row M B01 ... M B15 and so on until row F.
Pos column is irrelevant at the moment.
MdCod column represents a code of the machine being added to the position in the certain module. It can be replaced by another machine at any given time.
I have one query that will be inserting data into this table by copying entries from another table, every time a new machine is added to one of the positions.
Tricky part for me is a second query that should be comparing records in two phases and if:
1) Inside same module (first pass of the query represented with red color in the example pic attached):
ModNam value is the same, MdCod matches between the entries then the most recent datestamp decides the single one to stay and others duplicates get deleted
2) Inside other module (second pass of the query represented with purple color in the example pic attached):
ModNam values are different and MdCod matches between the entries then the most recent datestamp decides the single one to stay and others duplicates get deleted.
Please help and advise.
Example pic (updated):
Thank you all in advance.

List of standard lengths for database fields

I'm designing a database table and asking myself this question: How long should the firstname field be?
Does anyone have a list of reasonable lengths for the most common fields, such as first name, last name, and email address?
I just queried my database with millions of customers in the USA.
The maximum first name length was 46. I go with 50. (Of course, only 500 of those were over 25, and they were all cases where data imports resulted in extra junk winding up in that field.)
Last name was similar to first name.
Email addresses maxed out at 62
characters. Most of the longer ones
were actually lists of email
addresses separated by semicolons.
Street address maxes out at 95
characters. The long ones were all
valid.
Max city length was 35.
This should be a decent statistical spread for people in the US. If you have localization to consider, the numbers could vary significantly.
UK Government Data Standards Catalogue details the UK standards for this kind of thing.
It suggests 35 characters for each of Given Name and Family Name, or 70 characters for a single field to hold the Full Name, and 255 characters for an email address. Amongst other things..
W3C's recommendation:
If designing a form or database that will accept names from people
with a variety of backgrounds, you should ask yourself whether you
really need to have separate fields for given name and family name.
… Bear in mind that names in some cultures can be quite a lot longer
than your own. … Avoid limiting the field size for names in your
database. In particular, do not assume that a four-character
Japanese name in UTF-8 will fit in four bytes – you are likely to
actually need 12.
https://www.w3.org/International/questions/qa-personal-names
For database fields, VARCHAR(255) is a safe default choice, unless you can actually come up with a good reason to use something else. For typical web applications, performance won't be a problem. Don't prematurely optimize.
Some almost-certainly correct column lengths
Min Max
Hostname 1 255
Domain Name 4 253
Email Address 7 254
Email Address [1] 3 254
Telephone Number 10 15
Telephone Number [2] 3 26
HTTP(S) URL w domain name 11 2083
URL [3] 6 2083
Postal Code [4] 2 11
IP Address (incl ipv6) 7 45
Longitude numeric 9,6
Latitude numeric 8,6
Money[5] numeric 19,4
[1] Allow local domains or TLD-only domains
[2] Allow short numbers like 911 and extensions like 16045551212x12345
[3] Allow local domains, tv:// scheme
[4] http://en.wikipedia.org/wiki/List_of_postal_codes. Use max 12 if storing dash or space
[5] http://stackoverflow.com/questions/224462/storing-money-in-a-decimal-column-what-precision-and-scale
A long rant on personal names
A personal name is either a Polynym (a name with multiple sortable components), a Mononym (a name with only one component), or a Pictonym (a name represented by a picture - this exists due to people like Prince).
A person can have multiple names, playing roles, such as LEGAL, MARITAL, MAIDEN, PREFERRED, SOBRIQUET, PSEUDONYM, etc. You might have business rules, such as "a person can only have one legal name at a time, but multiple pseudonyms at a time".
Some examples:
names: [
{
type:"POLYNYM",
role:"LEGAL",
given:"George",
middle:"Herman",
moniker:"Babe",
surname:"Ruth",
generation:"JUNIOR"
},
{
type:"MONONYM",
role:"SOBRIQUET",
mononym:"The Bambino" /* mononyms can be more than one word, but only one component */
},
{
type:"MONONYM",
role:"SOBRIQUET",
mononym:"The Sultan of Swat"
}
]
or
names: [
{
type:"POLYNYM",
role:"PREFERRED",
given:"Malcolm",
surname:"X"
},
{
type:"POLYNYM",
role:"BIRTH",
given:"Malcolm",
surname:"Little"
},
{
type:"POLYNYM",
role:"LEGAL",
given:"Malik",
surname:"El-Shabazz"
}
]
or
names:[
{
type:"POLYNYM",
role:"LEGAL",
given:"Prince",
middle:"Rogers",
surname:"Nelson"
},
{
type:"MONONYM",
role:"SOBRIQUET",
mononym:"Prince"
},
{
type:"PICTONYM",
role:"LEGAL",
url:"http://upload.wikimedia.org/wikipedia/en/thumb/a/af/Prince_logo.svg/130px-Prince_logo.svg.png"
}
]
or
names:[
{
type:"POLYNYM",
role:"LEGAL",
given:"Juan Pablo",
surname:"Fernández de Calderón",
secondarySurname:"García-Iglesias" /* hispanic people often have two surnames. it can be impolite to use the wrong one. Portuguese and Spaniards differ as to which surname is important */
}
]
Given names, middle names, surnames can be multiple words such as "Billy Bob" Thornton, or Ralph "Vaughn Williams".
I would say to err on the high side. Since you'll probably be using varchar, any extra space you allow won't actually use up any extra space unless somebody needs it. I would say for names (first or last), go at least 50 chars, and for email address, make it at least 128. There are some really long email addresses out there.
Another thing I like to do is go to Lipsum.com and ask it to generate some text. That way you can get a good idea of just what 100 bytes looks like.
I pretty much always use a power of 2 unless there is a good reason not to, such as a customer facing interface where some other number has special meaning to the customer.
If you stick to powers of 2 it keeps you within a limited set of common sizes, which itself is a good thing, and it makes it easier to guess the size of unknown objects you may encounter. I see a fair number of other people doing this, and there is something aesthetically pleasing about it. It generally gives me a good feeling when I see this, it means the designer was thinking like an engineer or mathematician. Though I'd probably be concerned if only prime numbers were used. :)
These might be useful to someone;
youtube max channel length = 20
facebook max name length = 50
twitter max handle length = 15
email max length = 255
http://www.interoadvisory.com/2015/08/6-areas-inside-of-linkedin-with-character-limits/
I wanted to find the same and the UK Government Data Standards mentioned in the accepted answer sounded ideal. However none of these seemed to exist any more - after an extended search I found it in an archive here: http://webarchive.nationalarchives.gov.uk/+/http://www.cabinetoffice.gov.uk/govtalk/schemasstandards/e-gif/datastandards.aspx. Need to download the zip, extract it and then open default.htm in the html folder.
+------------+---------------+---------------------------------+
| Field | Length (Char) | Description |
+------------+---------------+---------------------------------+
|firstname | 35 | |
|lastname | 35 | |
|email | 255 | |
|url | 60+ | According to server and browser |
|city | 45 | |
|address | 90 | |
+------------+---------------+---------------------------------+
Edit: Added some spacing
Just looking though my email archives, there are a number of pretty long "first" names (of course what is meant by first is variable by culture). One example is Krishnamurthy - which is 13 letters long. A good guess might be 20 to 25 letters based on this. Email should be much longer since you might have firstname.lastname#somedomain.com. Also, gmail and some other mail programs allow you to use firstname.lastname+sometag#somedomain.com where "sometag" is anything you want to put there so that you can use it to sort incoming emails. I frequently run into web forms that don't allow me to put in my full email address without considering any tags. So, if you need a fixed email field maybe something like 25.25+15#20.3 in characters for a total of 90 characters (if I did my math right!).
I usually go with:
Firstname: 30 chars
Lastname: 30 chars
Email: 50 chars
Address: 200 chars
If I am concerned about long fields for the names, I might sometimes go with 50 for the name fields too, since storage space is rarely an issue these days.
If you need to consider localisation (for those of us outside the US!) and it's possible in your environment, I'd suggest:
Define data types for each component of the name - NOTE: some cultures have more than two names! Then have a type for the full name,
Then localisation becomes simple (as far as names are concerned).
The same applies to addresses, BTW - different formats!
it is varchar right? So it then doesn't matter if you use 50 or 25, better be safe and use 50, that said I believe the longest I have seen is about 19 or so. Last names are longer

Resources