EGTB file splitting

[Martin Kreuzer]

Hi all,

does anybody know to to produce EGBT where
the *.nbb and *.nbw files are split into several parts?
So far I have been using

datacomp e:8192 name.nbb

to compress the nbb (and nbw) files I have computed.
But what if the resulting *.emd files get too big?
How do I produce a multipart EGTB?

Thanks for any hints you have!
Servus,
Martin Kreuzer

[mbourzut]

There was a tbsplit utility available in one of the Nalimov code releases.

The function is extremely simple. Start with the uncompressed .nb? files, and sequentially split them into files of size 2^31, with extensions .0.nb?, .1.nb?, etc.

-Marc

[Martin Kreuzer]

Dear Marc,

thank you very much for this info! I had already noticed that
by combining the .0.nbb , .1.nbb etc. files using
copy (in Windows) or cat (in Linux) I could create a file
with the exact length given in tbindex.cpp

Does anybody have the source code of tbsplit?
Greetings,
Martin Kreuzer

[mbourzut]

#include <stdlib.h>
#include <stdio.h>

#define CHUNK_SIZE (128ul*1024ul*1024ul)
#define FILE_SIZE (65536ul*32768ul)

int main (int argc, char *argv[])
{
FILE *fpIn, *fpOut;
char *pbBuf, *pchSrc, *pchDst;
char szFile[1024];
int iFile, fDot;
long cbRead, cbWritten;
unsigned i;

if (2 != argc && 3 != argc) {
printf ("Usage: tbsplit file_name [drive]\n");
return 1;
}
fpIn = fopen (argv[1], "rb");
if (NULL == fpIn) {
printf ("Unable to open \"%s\"\n", argv[1]);
return 1;
}
pbBuf = (char *) malloc (CHUNK_SIZE);
if (NULL == pbBuf) {
printf ("Unable to allocate %lu bytes of memory\n", argv[1]);
fclose (fpIn);
return 1;
}
for (iFile = 0;; iFile ++) {
pchSrc = argv[1];
pchDst = szFile;
if (3 == argc) {
strcpy(szFile, argv[2]);
pchDst += strlen(szFile);
}
fDot = 0;
for (;;) {
*pchDst = *pchSrc;
if (!fDot && '.' == *pchSrc) {
fDot = 1;
if (iFile < 10)
* ++ pchDst = (char) ('0' + iFile);
else
* ++ pchDst = (char) ('a' + iFile - 10);
* ++ pchDst = '.';
}
if (!*pchSrc)
break;
pchSrc ++;
pchDst ++;
}
fpOut = fopen (szFile, "wb+");
if (NULL == fpOut) {
printf ("Unable to create \"%s\"\n", szFile);
fclose (fpIn);
return 1;
}
for (i = 0; i < FILE_SIZE/CHUNK_SIZE; i ++) {
cbRead = fread (pbBuf, 1, (long) CHUNK_SIZE, fpIn);
if (0 == cbRead)
break;
cbWritten = fwrite (pbBuf, 1, cbRead, fpOut);
if (cbRead != cbWritten) {
printf ("Unable to write %lu bytes into \"%s\"\n", cbRead, szFile);
fclose (fpIn);
fclose (fpOut);
return 1;
}
if (cbRead != (long) CHUNK_SIZE)
break;
}
if (0 != fclose (fpOut)) {
printf ("Unable to close \"%s\"\n", szFile);
fclose (fpIn);
return 1;
}
if (cbRead != (long) CHUNK_SIZE)
break;
}
fclose (fpIn);
return 0;
}

[Martin Kreuzer]

Hi Marc,

thank you very, very much!

Greetings,
Martin Kreuzer