Operating System - OpenVMS
1748044 Members
4904 Online
108757 Solutions
New Discussion юеВ

Re: Any known problems with SYMLINKS ?

 
Shriniketan Bhagwat
Trusted Contributor

Re: Any known problems with SYMLINKS ?

Hi Steven Schweda,

Please provide more information on the issues you are facing with BACKUP. Does the file reporting BACKUP-E-OPENIN error is a symlink file? It would be good, if we have some kind of reproducer to reproduce the issue locally.

Regards,
Ketan
Shriniketan Bhagwat
Trusted Contributor

Re: Any known problems with SYMLINKS ?

Hi,

One more info.

BACKUP treats symlinks as ordinary files. They are saved and restored as is, but are not followed. Consequently, BACKUP will not follow symlinks in its searching of directory paths either. This is how the current BACKUP├в s designed WRT symlink.

Regards,
Ketan
P Muralidhar Kini
Honored Contributor

Re: Any known problems with SYMLINKS ?

Steven,

>> Does BACKUP /COMPARE work yet? On my V8.3
>> Alpha system, I tend to get stuff like:
>> %BACKUP-E-OPENIN, error opening ALP$DKA0:[UTILITY.SOURCE.OPENSSL.openssl-1_0_0d.test]md5test.c;1 as input
>> -RMS-F-ORG, invalid file organization value

This problem is seen with V83 version.
However the SYMLINK redesign in V84 has fixed this problem. I have attached the logs of BACKUP/COMPARE command with SYMLINK on a V83 and V84 system.

Regards,
Murali
Let There Be Rock - AC/DC
Robert Gezelter
Honored Contributor

Re: Any known problems with SYMLINKS ?

Ketan,

Re: BACKUP and symlinks

On the question of following symlinks during a BACKUP operation, arguments can be made for/against.

However, with all due respect (WADR) there is a need both ways for appropriate warnings. For example:

- symlink included, but not followed:

BACKUP-W-SYMNOTFLW Symbolic link included in file specifier; not followed.

- symlink included, but followed:

BACKUP-W-FILRESTOUT File restored outside specifier from symbolic link

The problem here is compliance and auditing. If someone in all innocence, attempts a backup, and because of a symbolic link does not actually backup the data, there could be a problem at a later date, with potentially serious consequences. Conversely, if something is restored outside of the expected path, BACKUP needs to report it so it can be logged for posterity.

Consider the corporate scenarios of what happens when someone attempts to preserve/restore a directory clade before/after an update or test. If a file that would be expected to be referenced is preserved or not preserved can be a source of serious problems. In the pressure of the moment, the semantic distinction between having a file and only having a symlink to a file can be lost. This can cause a great deal of confusion in what is often a time of high stress. Human factors requires clarity for accurate decision making.

This is similar to the classic programming problem of damaging an input variable because it is call-by-reference instead of call-by-value. People who work in several different languages, each with its own default behaviors (e.g., FORTRAN/C) are familiar with the symptoms of this problem.

- Bob Gezelter, http://www.rlgsc.com
Shriniketan Bhagwat
Trusted Contributor

Re: Any known problems with SYMLINKS ?

Hi Bob,

Thanks for the input. I agree with you. It is better to notify the user than being silent.

Regards,
Ketan
Hoff
Honored Contributor

Re: Any known problems with SYMLINKS ?

What happens when you restore one of these BACKUP savesets, particularly when the saveset is populated with some unknown number of symlinks?

Do you get to locate and resolve an unknown number of now-dangling symlinks to location(s) that may not or do not exist in the restored configuration?

I'm guessing I now have to post-process the restored volume, looking for and updating or removing these symlinks.

Then there are the potentially very obscure results of restarting an application on a restored volume on a backup server, and the application encounters a latent symlink pointing off into hyperspace. Particularly when you're in crunch-mode working to get the application back online.
Craig A Berry
Honored Contributor

Re: Any known problems with SYMLINKS ?

The DIRECTORY command got the negatable /SYMLINK qualifier; it seems to me the BACKUP command is even more in need of fine-grained file selection. It's not immediately obvious to me what the default should be, though, nor that it should be the same for every type of backup. /PHYSICAL and /IMAGE should probably not follow links by default.

And then there is the problem of what to do on restore when the file the symlink points to belongs in a place that doesn't exist on the restore target (such as on another volume that isn't there).

I can imagine designing something workable would be non-trivial. But doing nothing doesn't seem entirely workable either.
P Muralidhar Kini
Honored Contributor

Re: Any known problems with SYMLINKS ?

Robert,

>> Thank you for the citation, I did not have the time when I posted that
>> comment to dig through things for the citation.
Thanks Craig for the link.
Yes, the information about the SYMLINK feature and the need to run VERIFY from
V84 node to convert the SYMLINK created on V83 node in to a format compatible
with V84 node is mentioned in the OPenVMS V84 Release notes.

>> Since there is an operational incompatibility, the upgrade procedure NOT JUST
>> THE DOCUMENTATION should warn people that there needs to be an
>> ANALYZE/DISK_STRUCTURE performed.
Yes, thats a good suggestion.
Something like the upgrade procedure should display a message as to what
needs to be done in case there are SYMLINKS already created from an pre V84
system.

I will keep these things in mind. Something to consider when doing any similar
kind of work in the future.

>> - symlink included, but not followed:
>> BACKUP-W-SYMNOTFLW Symbolic link included in file specifier; not followed.
>>
>> - symlink included, but followed:
>> BACKUP-W-FILRESTOUT File restored outside specifier from symbolic link
>>
>> The problem here is compliance and auditing. If someone in all innocence,
>> attempts a backup, and because of a symbolic link does not actually
>> backup the data, there could be a problem at a later date, with potentially
>> serious consequences
Thats a excellent suggestion.
Currently i think the BACKUP copies only the SYMLINK and does not report any
message indicating whether the target of SYMLINK is copied or not.
Displaying a message about this would alert the system administator, so that he
can take the necessary action. If the target also gets copied as a part of the
BACKUP then its ok, otherwise the symlink would be pointing to nothing (or to
a wrong file that already exisits) when it gets restored at some future date.

I am sure Ketan would have made a note of this in his to-do list for BACKUP utility.
I will check with him anyways.

Hoff,
>> What happens when you restore one of these BACKUP savesets, particularly
>> when the saveset is populated with some unknown number of symlinks?
when taking a BACKUP, the SYMLINKS are treated as ordinary files and are
copied as it is. i.e. the target is not followed. When BACKUP saveset is restored,
the SYMLINKS would be created on the disk.

* If the target happened to be copied as a part of the BACKUP saveset then the
restored SYMLINK would work. If not then the following is possible

* It may happen that there is some other file already present which corresponds
to the same path as referenced by the SYMLINK.
In this case the acessing the SYMLINK would end up accessing that file.
or
* SYMLINK would be dangling. Its link corresponds to a file that was not restored
as a part of the BACKUP and hence does not exist. Access to the SYMLINK would fail.

>> Do you get to locate and resolve an unknown number of now-dangling
>> symlinks to location(s) that may not or do not exist in the restored
>> configuration? I'm guessing I now have to post-process the restored volume,
>> looking for and updating or removing these symlinks.
Yes, i guess thats the way. Looks like a lot of manual work is involved here.
Need to locate all the symlinks that got restored. Check their target (which could
again be another symlink). The target may not be present in the restored
configuration as the target could reside on some other disk and so on.

>> Then there are the potentially very obscure results of restarting an application
>> on a restored volume on a backup server, and the application encounters a
>> latent symlink pointing off into hyperspace.
>> Particularly when you're in crunch-mode working to get the application back
>> online
Yes, thats correct. These are some excellent scenarios that we need to make a
note when talking about BACKUP and SYMLINKS. I am not sure if this is
documented, if not then i guess it would be worth documenting it.

Thanks everyone for the comments. There are so many things to consider when
we look from the customers perspective. Good learning.

Regards,
Murali
Let There Be Rock - AC/DC
Hoff
Honored Contributor

Re: Any known problems with SYMLINKS ?

An update to BACKUP /INTERCHANGE might be a reasonable way to cause BACKUP to flag or to strip off all off-disk or out-of-purview of the BACKUP symlinks. It already does something sort-of similar, stripping ACLs.
Robert Gezelter
Honored Contributor

Re: Any known problems with SYMLINKS ?

Murali (and Ketan),

Concerning my earlier comment about ANALYZE/DISK_STRUCTURE (aka VERIFY).

If I may, a suggestion for a standing REQUIREMENT for symlinks, and any similar structural change that will result in a forward incompatibility:

- a command procedure to be run before (emphasis: BEFORE) upgrading to ensure readiness for upgrade.

- the command procedure should have two modes: identify and update. Identify merely lists the items that would need to be changed.

This will prevent errors leading to numerous support calls.

- Bob Gezelter,http://www.rlgsc.com