mysql

[Bojan Nemec]

Hi,

A few days ago I look to the mysql source distribution from the freeware CD. I realise that it is possible to write a custom storage engine. My idea is to write a storage engine which works with RMS files (index files for begining) and supports the VMS data types so I can access to old RMS files with SQL.
First tests looks good. The mysql storage engine fits to RMS without many problems.

Now the questions:

Has anybody try to do a similar work?

Anybody knows what is with the VMS port for mysql 5.1?

Bojan

[Jean-François Piéronne · Accepted solution]

Hi,

I have had some requests for a RMS engine but I haven't done any work.

Do you plan to release your RMS engine ?

The current 5.1 port on VMS is 5.1.11, I have planned to port the latest 5.1, but, yet, no schedule.

JF

[Hoff]

RMS connector: donno. Haven't heard of one specifically for MySQL. There are certainly ODBC connectors for RMS. Probably JDBC.

MySQL 5.1 beta (old discussions):

http://dba.openvms.org/phorum/list.php?9

http://www.openvms.org/stories.php?story=06/04/22/5056423

Contact info for SysGroup SARL was included.

In either case, Jean-François Piéronne (jfp) would (also) be a likely contact.

[Bojan Nemec]

Jean-François,

As I said, I am only on the first steps. I done an engine rewriting the ha_example engine. It can read,write,update and can use indexes, it can also do some data conversions from mysql to VMS data types and viceversa. For now no locking, table creation etc...

If I succed in my implementation I will certainly release the code.

I am intersted in any 5.1 version. The storage engine specification is slightly different in this versions. As I see there are some more functions in the "handlerton" and it will probably be possible to use transactions when RMS Journaling is available.

It is possible to somehow get yours last port?

Hoff,

Thanks for the links, I found them when I was searching for the release.

Bojan

PS
10 points for Jean-François are for his great work on mysql and other freeware port to VMS.

[Jean-François Piéronne]

Bojan,

I will try to finish the port of MySQL 5.1.14, I have started this port a month or two ago but I did not finish due to the lack of time. Then I will put it online.

I can also put online 5.1.11 which is a MySQL 5.1 beta but which seem to work fairly well as some VMS sites are running this version for near a year without many problems.

JF

[Bojan Nemec]

Jean-Francois,

For my experiments the beta version will be OK. Probably better than working on the old MySQL 4.1.14.

I will appreciate if you can put online MySQL 5.1.11.

Thanks in advance.

Bojan

[Jean-François Piéronne]

Bojan,

I have put online 5.1.11, same location as 4.1.14 kit.
I will try to finish the port of 5.1.14 next month.

JF

[Bojan Nemec]

Jean-Francois,

Thanks for putting online 5.1.11, I have downloaded it.
Now I have to see which are the differences betwen the old version an the new one.

Thanks one more time

Bojan

[Robert Atkinson]

Bojan, how is the definition for the existing RMS file stored, i.e. field types, sizes, etc?

Rob.

[Bojan Nemec]

Robert,

As I said this are my first steps. For now I have put the descriptions of the existing fields in the comment string of each field.
In the comment you put the size,type,class (string or decimal scalar string) and optional scale and digits from the decimal scalar string.

This comment is parsed when the file is open and VMS descriptors are made for fields which need conversion. When the field is transfered from MySQL or to MySQL LIB$CVT_DX_DX is called. In the future maybe some more optimal method will be used.


Now I downloaded the 5.1.11 version and have some troubles in compiling from sources.

Bojan

[Jean-François Piéronne]

Bojan,

what problems do you have ?

I was able to rebuild from sources without any problem.


JF

[Bojan Nemec]

JF,

I attach the errors from the build procedure.
The build procedure (in this case) was:

$ @logicals
$ @symbols
$ @build_all JFP_SSL ZLIB SERVER LONG MIXED

But I have try also with:

$ @build_all VMS_SSL ZLIB SERVER SHORT MIXED
$ @build_all JFP_SSL ZLIB SERVER SHORT MIXED

I hae also tried to exclude the ha_example from the final link. But it seems that most of objects located in MYSQL051_ROOT:[vms.mysql.o_alpha.sql] are missing (there are only two my_lock.OBJ and net_serv.OBJ).

Bojan

PS
Maybe my compilers versions are to old?

$ cxx/version
Compaq C++ V6.5-004 for OpenVMS Alpha V7.3-2
$ cc/version
Compaq C V6.5-001 on OpenVMS Alpha V7.3-2

[labadie_1]

I think you should get C and C++ 7.1 or 7.2, earlier versions fail.

[Jean-François Piéronne]

Bojan,

To build the server you have to run
1. logicals.com
2. build_server.com, you may have to update this procedure which contains
$ @python_vms:setup
$ set def mysql051_root:[vms.mysql]
$ python genbuild_mms.py server 32 mixed
$ set def mysql051_root:[vms]
$ @build_all JFP_SSL ZLIB SERVER SHORT MIXED


You need to execute the python script to generate the correct mms files.

Sometimes I will build a separate tree for the server and all the clients but, currently, this share the same tree structure, so it is necessary to clean all object and libraries (olb) before the build.

I have somewhere a procedure which do all the job (build the server and the various clients).
But to only build the server you just need the 2 procedures logicals.com and build_server.com

JF

[Bojan Nemec]

JF,

I will try to generate mms files but first I have to install Python (it seems that the installation needs some time).

Thanks
Bojan

[Jean-François Piéronne]

Bojan,

Install python is very easy, you just have to download and mount 2 ld images, you probably can probably install Python in a couple of minutes.
More informations:
http://vmspython.dyndns.org/DownloadAndInstallationPython

JF

[Bojan Nemec]

JF,

Just in the process of downloading the ld images. When I try to install python from PCSI it seems to go in an infinite loop after the line:
Compiling /python_root/local/rpdb2.py ...

Thanks

Bojan

[labadie_1]

If you just want to use Movable Python, once you have the two .dsk files

1) if you want Python just for your process

$ ld connect jfppy0006/symbol
$ mount $1$lda'ld_unit' jfppy0006
$ @ $1$lda'ld_unit':[000000]python_logicals
$ ld connect jfplib0001/symbol
$ mount $1$lda'ld_unit' jfplib0001
$ @ $1$lda'ld_unit':[000000]lib_logicals
$ @python_vms:setup
$ python python_root:beer.py


2) if you want Python available for all processes


$ ld connect jfppy0006/symbol
$ mount/system $1$lda'ld_unit' jfppy0006
$ @ $1$lda'ld_unit':[000000]python_logicals
$ ld connect jfplib0001/symbol
$ mount/system $1$lda'ld_unit' jfplib0001
$ @ $1$lda'ld_unit':[000000]lib_logicals
$ @python_vms:setup
$ python python_root:beer.py

[labadie_1]

Sorry, I made a mistake, the second part should be

2) if you want Python available for all processes


$ ld connect jfppy0006/symbol
$ mount/system $1$lda'ld_unit' jfppy0006
$ @ $1$lda'ld_unit':[000000]python_logicals
$ ld connect jfplib0001/symbol
$ mount/system $1$lda'ld_unit' jfplib0001
$ @ $1$lda'ld_unit':[000000]lib_startup
$ @python_vms:setup
$ python python_root:beer.py

[Bojan Nemec]

I succesfuly installed the movable ld images and executed the mms file creation.
Now the build_all command procedure is building the server image. I work on a slow alpha workstation 600au and this will take time. For now only a few informationals and two warnings.

Thanks
Bojan

[Bojan Nemec]

After some smaller problems I finaly build the server (mysqld.exe) and run the first setups. For now it seems to work OK.

The problems were:
STRING_WITH_LEN macro in the sql_select.cc in memcpy -> changed the source from
memcpy(table_name_buffer, STRING_WITH_LEN("to
memcpy(table_name_buffer, "
Some symbols were missing when linking ->
added these two lines
SSLLIB:LIBSSL_MIXED.OLB/LIB
SSLLIB:LIBCRYPTO_MIXED.OLB/LIB
to the mysql.opt file (I dont know why but the build_all command procedure does not append these lines)

uncomment 3 lines in mysql.opt:
MYSQL051_ROOT:[vms.mysql.cxx_repository]CXX$bnlgprprpndngrwsvnt30ktmg87
MYSQL051_ROOT:[vms.mysql.cxx_repository]CXX$bnlgprprpndngrwsvnt30qa0oec
MYSQL051_ROOT:[vms.mysql.cxx_repository]CXX$bnlgprprpndngrwsvnt328op6i8

Now its all on me. As brian wrote in the ha_example.cc:

Happy coding!

Bojan

PS
Labadie,
It seems that is possible to compile the sources with the older versions of compilers.

[Jean-François Piéronne]

Building the server on my system (VMS 7.3-2 cc 7.2, cxx 7.2)

sql_select.cc compile without any problem.

May be due to a compiler difference.


End of the opt file:
MYSQL051_ROOT:[vms.mysql.o_alpha.vio]viosslfactories.OBJ
MYSQL051_ROOT:[vms.mysql.o_alpha.archive]azio.OBJ
LIBZ_SHR32/share
SSLLIB:LIBSSL_MIXED.OLB/LIB
SSLLIB:LIBCRYPTO_MIXED.OLB/LIB
sys$share:LIBCXXSTD.OLB/lib

So the ssl lib are correctly include.

Also the 3 lines are still comment out.

And the link just return a duplicate symbol:
%LINK-W-MULDEF, symbol DECC$__DL__XPV multiply defined
in module DECC$SHR_EV56 file SYS$COMMON:[SYSLIB]DECC$SHR_EV56.EXE;1

So, if you can, it would be a good option to upgrade your compilers. MySQL is a good stress test for compilers :-)

JF

[Bojan Nemec]

JF,

Done some testing.

Seems that memcpy in my version of CXX is defined as a macro. The compiler complains too few arguments in macro invocation.

The SSL libraries are not appended in the build_all command procedure so this has nothing to do with the compiler. I have not done further investigation.

The compiler also give me a warrning abbout compatibility with a previous declaration for the 3 commented lines.

At the end the linker return the same duplicate symbol.

For now I will try to stay without upgrading the compiler.

I think that I have to rewrite the link procedure to made mysqld.exe a shareable image.

Thanks

Bojan

[Martin Vorlaender]

JF,

more bad news...

Reading this thread, I thought I'd try to build it on I64 V8.3. But the build procedure stumbles over the following error (multiple times):

char *crypt (const char *__key, const char *__salt);
......^
%CC-E-NOTEXPECTING, Error parsing parameter list. Found "(" when expecting one of: , , "...", ")".
at line number 406 in module UNISTD of text library SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1

The cause for this is mysql051_root:[vms.mysql]upcase_names.h :

#define crypt(b,s) DES_crypt((b),(s))

in conjunction with unistd.h :

#if __CRTL_VER >= 80300000
void encrypt(char __block[64], int edflag);
char *crypt (const char *__key, const char *__salt);
#endif




cu,
Martin

[Jean-François Piéronne]

Martin,

You may try the following patch (untested):
--- upcase_names.h;-2 Mon Apr 24 21:49:38 2006
+++ upcase_names.h Thu Jun 21 22:32:24 2007
@@ -57,7 +57,7 @@

#ifndef SSL_NAMES_ARE_UPCASE
# include
-# if OPENSSL_VERSION_NUMBER >= 0x00907000L
+# if OPENSSL_VERSION_NUMBER >= 0x00907000L && __CRTL_VER < 80300000
# define crypt(b,s)\
DES_crypt((b),(s))
# endif
@@ -94,12 +94,14 @@
# define X509_get_subject_name X509_GET_SUBJECT_NAME
# define adler32 ADLER32
# define crc32 CRC32
-# if OPENSSL_VERSION_NUMBER >= 0x00907000L
-# define crypt(b,s)\
- DES_crypt((b),(s))
-# define DES_crypt DES_CRYPT
-# else
-# define crypt CRYPT
+# if __CRTL_VER < 80300000
+# if OPENSSL_VERSION_NUMBER >= 0x00907000L
+# define crypt(b,s)\
+ DES_crypt((b),(s))
+# define DES_crypt DES_CRYPT
+# else
+# define crypt CRYPT
+# endif
# endif
# define deflate DEFLATE
# define deflateEnd DEFLATEEND

I plan to upgrade next week one of my boxes (IA64) to 8.3, then I will check if MySQL (and Python) build correctly on this version.

JF