My name is Bruce Hayden and I work for IBM in the Advanced Technical Skills (ATS) organization on z/VM and Linux on System z. Let me get the normal disclaimer out of the way - Any statements or opinions expressed in this blog are my own. I do not speak for IBM. I do work for IBM and this blog is hosted by IBM, but I am not an IBM spokesman!
I started using IBM's VM operating system in college as a user and I've been a user and system programmer on VM throughout my entire IBM career, which is getting close to 30 years. During that time, I've built and maintained VM systems and done an extensive amount of system programming and automation on VM. My intent with this blog is to share some of the things that I've learned over the years, which could be hints on automation, setup, install, or whatever. I welcome suggestions for topics. I'll also post on Linux on System z since I do have experience with it and have worked with it quite a bit. My intent is to post a new entry about every 2 weeks, but that could change based on workload, vacations, or hot topics. This blog is not part of my normal job description, so interference from my "normal job" could slow down postings.
Thanks for reading!
A very good way to start off this blog would be to talk about the history of z/VM. But, I don't have to say much because an article was recently published in z/Journal magazine about this very subject. The article is titled The Last 30 Years of VM
by Neale Ferguson and it can be found at http://www.mainframezone.com/article/the-last-30-years-of-vm
. This article doesn't discuss the very beginnings of the VM operating system, but does discuss VM in its "glory days" and its transition to the VM we have today. I think it is important to have some background on z/VM so that you can better understand why VM is designed the way that it is and why some functions and features are included that seem to have limited use these days.
For even more history of VM, including its early history, you should look at the papers written by Melinda Varian. These can be found on her page at http://www.leeandmelindavarian.com/Melinda/index.html
. Scroll down for the "VM History" section to see the various papers in various formats, including e-book format!
Happy (history) reading!
The CPSYNTAX command has been around in VM for as long as we've had the SYSTEM CONFIG file. It is used to check the syntax of that file. The SYSTEM CONFIG file is only read by CP during IPL, and that is not a good time to discover that you have syntax errors in the file! Of course, the syntax checker can only check syntax, it can't tell you that you mistyped a device address or volume label. That is why you should keep backup copies of working config files or make sure you have recovery procedures if the system won't IPL correctly.
The SYSTEM CONFIG file has had the capability to define multiple systems in 1 file for a long time, but this capability was enhanced for SSI clusters. The System_Identifier and System_Identifier_Default statements (usually found at the top of the file) define a name for the system that can be referenced in the rest of the configuration file. (The "system identifier" may be called a system id, node id, or a member name.) This statement uses the model and CPU id or the LPAR name to associate a machine or LPAR to a system identifier. Then that name can be used to qualify statements in the SYSTEM CONFIG file that only apply to one system.
Anyway, this is all well and good, but when you run CPSYNTAX against a SYSTEM CONFIG file that defines multiple SSI members, the CPSYNTAX command requires you to specify the LPAR name or CPU id that will match one of the System_Identifier lines in the file. And, you will want to run CPSYNTAX against all of the SSI members that you have. I got tired of running CPSYNTAX multiple times to specify the LPAR name of each member, so I wrote the exec below and named it CPSYN EXEC. It reads all of the System_Identifier lines first and invokes CPSYNTAX for each member. Now, when I change SYSTEM CONFIG, I only need to enter CPSYN to do all of the syntax checks!
/* Run CPSYNTAX for all members of a SYSTEM CONFIG file */
/* Author: Bruce Hayden, IBM Advanced Technical Skills */
arg fn ft fm .
If fn='' then fn='SYSTEM'
If ft='' then ft='CONFIG'
If fm='' then fm='*'
'PIPE <' fn ft fm,
'| pick anycase w1 == "System_Identifier"',
'| stem sysids.'
If RC<>0 then
If sysids.0=0 then do
say 'No System Identifiers found.'
'EXEC VMLINK MAINT 193 (NONAMES NOTYPE PUSH'
Do i=1 to sysids.0
Parse upper var sysids.i . lpar lparname .
If lpar<>'LPAR' then do /* CPUID specified */
lparname = translate(lparname,'0','%') /* CPUID wildcard is % */
say 'Running CPSYNTAX for' lpar lparname
'EXEC CPSYNTAX' fn ft fm '('lpar lparname
If RC<>0 then
say 'Return code' RC 'for' lpar lparname
'EXEC VMLINK MAINT 193 (NONAMES NOTYPE POP'
If you have done any programming with z/VM, you are familiar with its diagnose codes. If not, here is a introduction to what they are, taken from the CP Programming Services document:
In a real processor,
the DIAGNOSE instruction performs processor-dependent diagnostic functions.
In a virtual machine, you use the DIAGNOSE interface to request that CP perform
services for your virtual machine. When your virtual machine attempts to
execute a DIAGNOSE instruction, control is returned to CP. CP uses information
provided in the code portion of the instruction to determine what
service it should perform. Once this service is provided, control returns
to the virtual machine.
Thus, because a DIAGNOSE instruction issued in a virtual machine results
only in returning control to CP and not in performing real-processor DIAGNOSE
functions, think of DIAGNOSE codes as a method of communicating virtual machine
requirements to CP.
There are many diagnose codes, and I'm sure later blog entries will discuss various ones. For this entry, I'm going to talk about diagnose x'A0'. (Diag codes are usually referred to by their hexadecimal numbers, therefore I'll refer to this code as A0, not 160.) This code is documented as "Obtain ACI Groupname" but that is only 1 of its functions. It has various subcodes to perform other functions, all of which are associated with running an external security manager (ESM) on your system or data normally associated with your security manager. Most of the subcodes of diag A0 are not documented because they are part of the interface to an ESM to CP and depend on what ESM you use. But a few are documented as programming interfaces.
Subcode 8 of diag A0 is used to test if an ESM is installed in CP or not. When you install z/VM, the ESM interface modules in CP are mostly "stub" modules that don't do anything. When you install an ESM, the ESM replaces the stub modules with its own code. Once you IPL with the ESM code in CP, an ESM must be active or many requests for resources or permissions will not be granted. Diag A0 subcode 8 can be used in a program to decide to take a different path when an ESM is active. I use this in AUTOLOG1 so it can decide if my ESM (RACFVM in my case) should be started or not. This lets me create a common system startup exec and not a different version for a system with RACF.
The problem with this is that the diag function in Rexx does not support diag A0. However, a very simple module can provide the information for subcode 8. My friend Alan Altmark shared some assembler code he had written long ago:
* Program to issue Diagnose X'A0', Subcode 8 (Query ESM)
* R15 = 0 : ESM installed
* 1 : ESM not installed
* Written by Alan Altmark, VM System Test 9/96
SLR R15,R15 R15 not used by diag
LA R1,8 Subcode 8 = ESM query
DIAG R0,R1,X'A0' Rx (R0) not used
When this code is run, it exits with return code 0 if an ESM is installed, and return code 1 if there is no ESM. To use this, paste the code in a file named DIAGA0 ASSEMBLE A and make sure it is in file format Fixed 80. You don't need the high level assembler to assemble this program. The assembler included with CMS works fine. Enter these commands:
SPOOL PRT TO *
VMLINK MAINT 193
VMFASM DIAGA0 DMSVM
LOAD DIAGA0 (CLEAR AMODE 24 RMODE 24 RLDSAVE
The "spool prt" command is entered because the VMFASM command will create a print file. The MAINT 193 disk is accessed because the macro libs and control files are on this disk. The GENMOD command will create DIAGA0 MODULE. To execute it, just enter DIAGA0. It will end normally if you have an ESM on your system, or end with return code 1 (the ready message will be "Ready(00001);") if you don't.
I use this in an exec with these lines:
'DIAGA0'Then I can code "If esmavail then ... " and take different paths in the code.
z/VM 6.2 adds another programming interface subcode to diag A0. It is subcode x'48' which returns a string that contains the ESM name, version, vendor, description, and the flag for an ESM active or not. It would really be nice if a Rexx interface to this subcode were provided!
This blog has been neglected lately because of vacations and travel, but I must add an entry for today (August 2, 2012) to wish VM a happy birthday! The VM product was announced on August 2, 1972 with VM/370, so today is when we consider VM to be "born". More information about the anniversary (ok, more people refer to today as its anniversary date and not its birthday) can be found at http://www.vm.ibm.com/vm40bday.html.
On the subject of web pages, I hope you're familiar with the VM home page
). It is the place you should go to find out more information about z/VM and its related products. Be sure to use the search facility
found on the sidebar on the right (named Site Search
). This is the easy way to find the web page you need.
The other page on the VM web site that I really like is the Downloads page (http://www.vm.ibm.com/download/
). This has a bunch of user written tools and utilities (457 of them today) that you can download and use. (Select the "packages page
" link to see the list.) Even if you can't find a specific package that does what you're looking for, you may find some ideas in other packages that are on that page.
Rexx on CMS (part of z/VM) contains some functions that the documentation calls "External functions and routines". These are functions that are not included in Rexx on other platforms, because they are specific to CMS and VM. One of them is CSL, for calling callable services library (CSL) routines and one is CMSFLAG which allows you to test several different flag values in CMS. The ones I use most often are DIAG and DIAGRC, which allow you to call diagnose codes directly from Rexx. As I described in an earlier blog entery, a diagnose instruction or diagnose call is really a hypervisor call in z/VM. In other words, it is a way for an operating system or program, running in a virtual machine, to ask the hypervisor (CP in our case) to do something for you.
The most useful diagnose code is diag 8, which lets you issue a CP command. This is how operating systems, including CMS, can issue a CP command. Not all diagnose codes that you find defined in the CP Programming Services book can be issued by the Rexx diag function, but there are a lot of useful ones. My real reason for this blog post is that the diag interface to CP is the true programming interface to CP. The output to CP commands is intended to be for humans to read, not programs. If a diagnose interface is provided to obtain information from CP, then it should be used instead of the output of commands! However, I agree that many times processing the output to a CP command is the only way to find out some setting or information, so that is what you must do.
For instance, the result returned from diagnose 0 is the "extended identification code". If you want to know the level of z/VM that you're running on, you should use this value, not the output of QUERY CPLEVEL. But, even code you find on your VM system has things like:
"PIPE CP QUERY CPLEVEL | STRIP | XLATE | VAR CPLEVEL"
Parse var cplevel . "VERSION" v "RELEASE" r "." m "," .
vrm = STRIP(v)||STRIP(r)||STRIP(m)
To obtain the "vrm" using diag 0 is actually less complicated and doesn't depend on keywords in the output:
vrm = v||r||m
It is true that you have to read the specifications of diag 0 to get the byte numbers, but it isn't that hard. Also, I used 5 lines to write this but it could also be written "vrm = c2d(substr(diag0,11,1)) || c2d(substr(diag0,37,1)) || c2d(substr(diag0,38,1))".
A bigger pet peeve of mine is using CMS pipelines to just suppress the output of a CP command. I see code like this: "PIPE CP XAUTOLOG VMSERVS" or "PIPE CP SLEEP 30 SEC". This is just as easy to read, runs faster, and achieves the same result: Call diag 8,'XAUTOLOG VMSERVS'. Rexx will put any output from the command in the variable "result" but you can just ignore that. Actually, the XAUTOLOG command isn't a good example because it has asynchronous output, so you're not really suppressing all the output. I code things like Call diag 8,'DETACH 120' frequently so that I don't see the "DASD 120 DETACHED" message. (Note: you can code this as "x=diag(8,'DETACH 120')" if you like, since it does the same thing as the call except assigns the output to "x" instead of "result".) To finish this though, "PIPE CP SLEEP 30 SEC" is really ridiculous, since the SLEEP command doesn't produce any output! Just code it as 'CP SLEEP 30 SEC'.
One other useful diagnose code that should be used instead of a CP command is diag 24 or 210 to get device information. A very good use of diag 24 is to test if your virtual machine is disconnected: If substr(diag(24,-1),13,1) = 2 then say "I am disconnected". I've also coded the test as "substr(diag(24,-1),13,1) <> 0".
I'll close this by wishing some of the new diagnose codes provided in recent releases of VM were supported by the Rexx diagnose interface. An earlier blog post talked about Diag A0 and an assembler routine to query it. It would be nice if Rexx had an interface to diag A0 subcodes 8 and 48. Also, and interface to diag 2cc (SSI query) would be useful, instead of having to parse the output from QUERY SSI!
The monitor service in CP (the z/VM control program) is what provides data to performance monitors or any other program that wants more details about what CP is doing. The main user of monitor data is for monitoring performance, using a product such as IBM's z/VM Performance Toolkit (also known as Perfkit.) The monitor is configured using the CP MONITOR command and then the performance monitor loads a special memory segment (usually named MONDCSS) to actually retrieve data from CP. Coordination with CP occurs using IUCV to communicate the monitor system service which is named *MONITOR. The CP MONITOR command sets an interval (called the monitor interval) which defines how often CP collects data from its counters and makes it available in the monitor segment. The default time for this interval is 1 minute.
All of the above it just background about what I wanted to post today. When you start the monitor, CP collects data for the monitor interval and then notifies any application that is connected to *MONITOR that new data is available. It keeps doing this until the monitor is stopped. I was asked once if there was a way to set the monitor intervals to start at the beginning of each minute. There isn't a direct way in CP but you can create an EXEC to wait for the start of the minute and then issue the CP MONITOR START command. Here is the code that I inserted in the PROFILE EXEC of the PERFSVM user id. This is the user that runs the Performance Toolkit.
/* Make the monitor intervals start on second zero */
'CP MONITOR STOP'
Parse value time('N') with hh ':' mm ':' ss .
mm=mm+1 /* Wait for the next minute */
If ss=59 then mm=mm+1 /* May need a bit more time */
If mm>60 then do /* Overflow to the hour */
'WAKEUP' hh':'right(mm,2,0)':00' /* Wait */
'CP MONITOR START' /* Start the monitor */
The code stops the monitor, queries the time, and then waits for the beginning of the next minute. When that occurs, it starts the monitor and from then on, every interval should start at the beginning of the minute.
The LINK command in CP has the syntax: LINK userid vdev1 vdev2 mode password, where userid can be "*" to indicate your own userid, The mode and password are optional parameters. If the link mode is omitted, it defaults to "RR". (There are a couple of keywords you can insert also, such as TO and AS to form a command such as "LINK TO MAINT 190 AS 490" if you like the command to be a bit more readable.) Many people are used to using only the link modes of RR or MR and may not realize that there are 12 valid link modes. When you are writing an EXEC that needs to link a disk, using the correct link mode will help you write more robust code. And understanding the link modes in creating directory entries (where there are even a few more valid letters you can add) and even working from the command line are things a good VM system programmer should know!
The "classic" link mode letters are R, W, and M. S (for stable) and E (for exclusive) were introduced in VM/ESA as part of the support for CP parameter (parm) disks. Let me just discuss the classic modes in this entry. R stands for read, W for write, and M for multiple. These seem to be straightforward, but the devil is in the details. The link modes on the LINK command can be 1 or 2 letters. The first letter can be R, W, or M and the 2nd letter can be R or W. The 2nd letter tells the LINK command what mode to link the disk as if the requested mode in the first letter fails. If you read the help for the single letter modes (R, W, or M) you'll see clauses like "the request is satisfied unless ...". It is those "unless" conditions that make make the LINK command fail the first attempt and go the second. The 2 letter link modes (just the "classic" ones still) do not have "unless" - they give you a link mode. (Once you read further, you'll see that in fact they really do have an "unless" because of stable and exclusive links, but don't worry about that right now.) So, a link request of "R" will fail if there is already another user with a read link. Using "RR" will always give you a read link. And so on.
You don't see link mode "W" used very much because it is only useful in some limited situations. It requests a write link, but that will fail if there is any other link to the disk. Mode "WR" will give you a read link if there is any other link. If you have an application that should only update a disk if nobody else is using it, then mode W is what you should use. But for CMS formatted disks, there are techniques to use to update a disk that other users are reading, so applications usually don't wait for all other links to be dropped first.
Link mode "M" is usually the one used to request a read/write link to a disk. It requests a write link, and only fails if there is another write link already established. This is a good thing, since it in general prevents disk corruption from multiple simultaneous writes to the same disk, which the CMS filesystem and most Linux filesystems don't support. Adding R to the request (link mode MR) requests a write link but falls back to a read link if there is already a write link. This is very handy for disks formatted with the CMS file system, since it is designed for sharing with many read only users. Adding the W to the request is the dangerous one (link mode MW). This establishes a write link no matter what. In other words, no check is made if any other user has a write link. Only use link mode MW when instructions tell you to do it, and that you're sure the application supports it.
One of my pet peeves is that I see many Linux machines and documentation for creating Linux machines that use mode MR in the minidisk statements in the directory. What is good for CMS use is not always good for Linux! Unless you set up Linux for it, it does not like having read only access to its disk(s)! Of course, if it is read only, the data on the disk is not changed but Linux won't run correctly. Some times this condition is not immediately apparent, especially if nobody is watching the Linux console as it boots up. A better way to set up Linux users is to use link mode M instead of MR. This way, the Linux machine will either get a read/write link to the disk or no link at all. If it doesn't get any link, Linux will usually fail to boot up instead of booting up to a crippled state.
I've also seen CMS applications, including code from IBM, use the wrong link mode. I've seen code that requests a link using MR, checks the return code, accesses the disk, and then has other code to see if it has read/write access to the disk. It would have been so much easier to request the link using mode M and check the return code. If it is zero, you have a read/write link! (I saw IBM supplied code this morning that had this "error" which prompted me to write this..)
I'll discuss the exclusive and stable modes in my next entry..
I discussed the more common CP link modes in my last entry. In this entry, I'll discuss the more uncommon ones for stable and exclusive links. These allow (as the names imply) a virtual machine to have a disk that is "stable" (unchanging) or have a disk that nobody else can use (exclusive.) Not just any user can request a stable or exclusive link. The authority can be granted for a single minidisk by adding the suffix S or E on the MDISK or LINK statement in the directory, or it can be granted for any links the virtual machine wants to make by adding the LNKSTABL and/or LINKEXCLU keywords to the OPTION statement in a user's directory entry.
These capabilities were added to VM/ESA when CP parm disks were added. It allowed CP to obtain a stable read link to a parm disk so that CP didn't have to worry about the files changing while it had a cached copy of the disk directory. If a virtual machine (or CP) has a stable read (SR) link to a minidisk, then no other virtual machine can obtain a write link to that disk. A stable write (SW) or stable mulitple (SM) link to a disk prevents any other virtual machine from obtaining a write link to the disk. The exclusive options do what they say. If a virtual machine has an exclusive read (ER) or exclusive write (EW) link to a disk, then no other virtual machine can obtain any other link to the disk. The most commonly used link modes in this set are SR (stable read) and EW (exclusive write). This assures a virtual machine that a disk won't change (SR) or if it is updating a disk, no other virtual machine will be affected (EW).
Most write-ups I've seen on using read only minidisks for Linux systems do not mention how these link modes could be used. Linux does not have methods like CMS does to update a disk while other virtual machines have read only links. So, I recommend that if you will share a read only disk with more than 1 Linux system, then have each Linux system link the disk with link more SR. That way you are assured that the disk will not be changed by some other virtual machine. So how do you update a disk that is defined this way? You need 2 disks, one that is the active disk and one that is not in use. That, of course, requires some other way of tracking which disk is the active one. Since Linux can read CMS files off of CMS formatted disks, it would not be too hard to create a "switch" that would indicated to both Linux and CMS (such as a PROFILE EXEC that runs on a Linux guest machine before Linux is started) which disk should be used.
I really like the functionality provided by Operations Manager for z/VM. I'm sure there will be some future blog entries about some of the ways I've used this product. Operations Manager provides scheduling functions that just about every z/VM system needs but one of the best things that it provides is an interface to be able to work with your disconnected service machines and Linux servers without needing to log on to the user id. You set up the main operations manager server (usually OPMGRM1) to be the SECUSER or OBSERVER of all of your disconnected machines, and it will watch for important messages that you define or let you view past messages. You can even enter commands that are sent to the disconnected machine to be executed.
The interface between your (authorized) virtual machine and the operations manager machine is a command named GOMCMD. The documentation says that to use this command, you first need to link and access the disk with the command, set up your virtual machine by issuing 'SET LANGUAGE (ADD GOM USER' and then issue GOMCMD OPMGRM1 <command>. The first 2 rules aren't too hard especially if you put the user interface files for operations manager on the Y disk and put the SET LANGUAGE in your PROFILE EXEC. But I really didn't like having to enter the name of the operations manager virtual machine OPMGRM1 each time I issued a command. Fortunately, GOMCMD is a module so I can easily make an exec front end for it on my A disk. As part of this I could use VMLINK to automatically link and access the disk with GOMCMD MODULE and issue the SET LANGUAGE if needed. Here is what I wrote:
/* Front end for GOMCMD MODULE */
arg svmarg command
If svmarg='?' then do
Address CMS 'HELP AGOM MENU'
'PIPE command QUERY LANGUAGE ALL',
'| strfind /GOM/',
'| count lines',
'| var loaded'
If loaded=0 then
'SET LANGUAGE ( ADD GOM USER'
When svmarg=svm then
'GOMCMD' svm command
'GOMCMD' svm svmarg command
I put GOMCMD and friends on my Y disk, so I don't have to link a disk, but it would be very easy to add EXEC VMLINK OPMGR (PUSH at the beginning and EXEC VMLINK OPMGR (POP at the end. With this exec, I can enter either GOMCMD <command> or GOMCMD OPMGRM1 <command>. The select statement is used in case I want to create other special commands. And, since I couldn't remember that I needed to type HELP AGOM MENU to get the help menu, I made entering GOMCMD ? take me to that menu.
One of the commands I like is VIEWCON (view a console), but I don't like the default syntax of this command either. The default syntax is GOMCMD OPMGRM1 VIEWCON USER(OPERATOR). (It doesn't have to be entered in upper case on the CMS command line but I usually show CP and CMS commands this way.) I'd rather just enter VIEWCON OPERATOR. So, I also have a VIEWCON EXEC:
/* Shortcut to GOMCMD VIEWCON ... */
When users='' then do
say 'No userids entered.'
When words(users)=1 & pos(',',users)=0 then
'EXEC GOMCMD VIEWCON USER('strip(users)')'
'EXEC GOMCMD VIEWCON USERS('translate(strip(users),',',' ')')'
You may notice that this exec uses the GOMCMD EXEC to invoke the GOMCMD MODULE. I did it this way so that if the user id of the operations manager machine is ever changed, I only have to update it in 1 place. The exec also supports the VIEWCON command syntax of GOMCMD OPMGRM1 VIEWCON USER(VMSERVR,VMSERVS,VMSERVU) to view multiple consoles on the same screen. In the exec, you can either separate the list of virtual machines by spaces or commas, such as VIEWCON VMSERVR VMSERVS VMSERVU.