Clocks-2 FTP Site

Endgame analysis using tablebases, EGTB generation, exchange, sharing, discussions, etc..
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Clocks-2 FTP Site

Post by clocks »

Hello Everyone,

I request help from everyone to bring the clocks-2 machine to having a full set of the 6-man files. It is on a COMPLETELY UNMETERED 10MBPS line. This means that a full 1250k/sec up is possible from this line. It is extremely reliable, and been up constantly (minus 1 short upgrade adding a 2nd 750GB drive).

With the upload of this machine, I am passing away the queue's very quickly, downloading over 300kb/sec consistantly on eMule. I have all of the 4+2 pawnless and pawnful files downloading this way.

If you have ANY complete 3+3/3+3P files I request you log in and upload them. The /Incoming folder has allows you to write files to it. Just browse the other folders and check out if the file is already available before you upload.

Server IP address: 75.126.91.34
Server Port: 21
User Name: EGTB
Password: EGTB

11GB of the 3+3 files are available
50GB of the 3+3P files are available
0.3GB of the 4+2 files are available
22.9GB of the 4+2P files are available
7GB 3-4-5 Files

You can access the server via your web browser using ftp://EGTB:EGTB@75.126.91.34/ if you don't have an FTP client software installed on your machine.

In the main directory there is a GREAT FTP client available for you to download (filezilla).

Additionally this machine is sharing about 30GB partial 4+2/4+2P on eMule. Any file added, after verified is shared by both eMule and by FTP to everyone.

It may not be so much yet, but with even just a couple more people working on it, the entire 3+3 set should be available entirely within just a few more days :)

Lets get this machine using its full upload 24/7. I hate paying so much for this dedicated line and watching it only uploading 300-400kb/sec on eMule to everyone at this point (largely due to having such a small amount of files so far < 150GB)

The site for the most part has a lot of the files from tablewell too. If you didn't finish them, they are available there. Of course download as many as you can. That is the whole reason I have this up. The site can distribute a full set in about 9 days. Lets use it! :)

Any comments, suggestions otherwise I am fully open to using. If there are any files anyone is wanting to distribute otherwise (pachnes IP filter, etc, etc) You are more than welcome to use this space as well. The server has 2 750GB drives, and I am happy to add more if we need!

Derek
User avatar
Kirill Kryukov
Site Admin
Posts: 7399
Joined: Sun Dec 18, 2005 9:58 am
Sign-up code: 0
Location: Mishima, Japan
Contact:

Post by Kirill Kryukov »

Hi Derek. This is incredible what effort you are putting into this! I am very happy not only because of your contribution, but also to see that this little project is able to gather so nice people. :-)

I will log in and start uploading some missing files, about weekend I think.
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Image

Stats so far, using about 25-30GB/day upload. Hoping to see this much higher in the coming days :) Wishing to see this at the full 95GB or so it can do each day. I think that puts it at a full set being handed out every 12 days. A little longer than I thought. If it gets to a point where it is being maxed every day, I can see what I can do about further upgrading it.

One idea is that the hosting company I use allows me to connect one server to another internally on their network. Although my bandwidth to the outside world is 10MBPS, I could run a 2nd server, accessing the first at a full 100MBPS, and have the same upload rate on the 2nd.

I'm having issues with getting a http server to run (at least with apache) on the machine. I found this option to be great, having partial downloads in my queue on eMule, but adding in a HTTP source (tablewell). This way not to waste upload capacities on tablewell redownloading a file I already had 40% completed or otherwise.

Once again, happy to hear any other ideas anyone might be having on what can be done here to best help everyone.

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Thanks Kirill :)

You can easily select the files that are most needed to be distributed first, maybe you can at least count the 1250k/sec upload as being 2 sources on your stats list on the main page ;) LOL

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Also to specify, you must put files you are uploading into the /incoming folder, no other folders have write access :)

Have noticed some wanting to upload files, but trying to put them into the correct directories themselves. I will add them myself after they are complete in the incoming folder.

I'm wanting to make sure there are no incomplete files in the main folder, and as these files are shared via eMule, someone possibly start downloading a partial files, etc.

Derek
User avatar
Kirill Kryukov
Site Admin
Posts: 7399
Joined: Sun Dec 18, 2005 9:58 am
Sign-up code: 0
Location: Mishima, Japan
Contact:

Post by Kirill Kryukov »

I have one idea for everyone uploading to clocks' FTP. 1. Upload only complete sets. 2. When you are going to upload a set, upoad an MD5 for that set first. This means you are going to upload that set, so when I see your MD5 file there I will know it and will upload other sets. I will also upload MD5 first. By the way this will also help Derek to check the incoming files.

One idea for Derek. Have you tried running multiple eMule clients on the same macnihe? I don't rememver now which mod can run multiple copies, Thomas should be able to tell that. By that way I think you should be able to saturate your connection with just one machine.
User avatar
Lutz Nebe
Posts: 18
Joined: Tue Jun 27, 2006 9:47 am
Sign-up code: 0
Location: Koblenz, Germany

Post by Lutz Nebe »

@ Derek Because TableWell ist currently down I started do upload KRR KRN (2 files) to your server.

@Kirill 426 files ( 160GB ) to go for the next complete set :)
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Thanks greatly for the upload :)

Actually speaking with another friend about what I am doing, turns out he may be able to get me even greater capacity than this line has.

For now this is great, but having the whole 6-man set hosted on a 100MBPS line for instance would be even better :)

My upload capacity from home is completely limited now by my ISP. Was really enjoying the 190k/sec up from here, but they capped me at about 50k/sec now. So adding in what I can too, but not nearly as quickly as I would be hoping.

I'll see what I can do in these next few weeks speaking with several others about getting these hosted on an even better line than this. I think it could to the point of 30-40 people downloading. If it were 40 people, than uploads would be at about 35k/sec. It helps, but looking for those 40 people to be getting minimum 100-200k/sec each.

I think a 100MBPS line is really what needs to be run :) LOL

Definitly wanting to "keep it in the family" here, and not shout out to the world about the FTP site only to get people that aren't going to reshare on eMule. :)

Derek
jkominek
Posts: 150
Joined: Mon Dec 04, 2006 9:02 am
Sign-up code: 0
Location: Pittsburgh, PA

Post by jkominek »

Derek,

I've taken your suggestion and have switched to FileZilla - and am enjoying it greatly.

Here is a suggestion for coordinating uploads for FileZilla users. First, FileZilla users select "skip" as the default upload action when a file already exists. Then for every verified emd file in your emule-shared directories, place a zero length file of the same name in /incoming (write a small script to do this). That way every contributor can brainlessly upload from their whole set - making tedious checking of existing files no longer necessary. (There are so many!)

john
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

I am completely unsure how to make a script to do this. It is an excellent idea though!

At this point there is not so many files, and I could do it manually, and easily keep up with after that point. Maybe it would take 1-2 hours for me to do?

If it is very simple, can someone write a script to do this? Just something I could run inside of each of the folders, and then create a 0-length folder in the incoming folder.

My setup on the hard drive is that it is in c:\TB (all folders are inside of that), some of the folders are just NTFS partitions from the 2nd 750GB drive that work the same as a regular folder to have this organized as best as possible. As I assume it could make a difference for the script.

I have complete (FULL) access to the machine, it is running windows 2003. Actually, I use it about 90% of the time as my own machine. Writing this post from it for instance. Anything is possible if there are other ideas anyone has.

Currently people are uploading at 200k/sec combined to the server over FTP. Emule is holding a consistant 300k/sec as a rough average. Honestly, I think this entire 6-man database could be online well within 1 month easily at this rate :)

And - looks like the entire 3+3 set within a matter of less than a week :)

Any 4+2P or 4+2 files uploaded, after verified are simply taken out of my eMule folder. I'm 'losing' 100MB here and there that was already downloaded, so for now best to upload the 3+3 / 3+3P files to me. I'm adding in any files that complete on eMule as well to these folders almost immediately as well :)

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Watching eMule here with the files people have been adding now being shared :)

eMuling has been going a steady 600k/sec. Thats about 2.1GB/hour being shared! Finally hitting the 50GB/day area now. :)

I just can't wait for 1 year to have passed by, and for the machine to have uploaded at least 30-40 full sets :)

I have one buyer in Michigan (I work on a pretty large-scale w/ laptop parts/accessories, etc) that may be able to offer me a quite a bit better line for this to be on. Would be just completely awesome to see a machine hitting 5MB/sec uploads on eMule :)

For now, I don't have anyone just leeching, and not giving anything back accross eMule. But, my real goal is for there to be 50-100+ people sharing the full sets accross eMule. 100 people with complete sets, and maybe another 50 in the process of downloading, combined with even a 3MB upload that I can do myself I think would put it in the range of 10MB/sec download rate for the entire network. It would mean a full set going out in less than 2 days. :)

Just my little project to really 'upset' this horrible download rate that I felt was wrong. Kirill said at some point I read on here, it just seemed wrong the files weren't available. I think its just really wrong it takes a year to get them ;) I'd really like to see it sub 1-month for anyone wanting them.

Derek
jkominek
Posts: 150
Joined: Mon Dec 04, 2006 9:02 am
Sign-up code: 0
Location: Pittsburgh, PA

Post by jkominek »

I've taken care of the first step for you. The incoming folder now contains mirror filenames for each emd file you have in the main shared directories. (Instead of making them zero length files, they have a small size.)

If you keep it up to date for us then FileZilla will skip over the files you already have complete. Note that I abandoned a couple partial file uploads, though.

By the way, you have:

33 - 41 files
33p - 132 files
42 - 10 files
42p - 105 files,
total: 288 files, 93.4 GB

Not a bad accumulation for such a short amount of time. It took me 10 months to finish my collection.

john
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Absolutely excellent! :)

Making the same sort of 'blanks' for anything else being added in.

I think pretty good 6 days to have this many files. LOL

eMule also has just shy of 38GB partial files, and I have maybe 1-2GB of files I need to add into the main directories from completed downloads.

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Seeing 450k/sec going out on eMule, and 350k/sec going out over FTP :)

Yep - I need a faster line when this set is complete LOL. I'll really work on this for everybody :)

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

I am noticing a couple of people downloading from the /Incoming folder.

I would prefer this not to be done. For one, I cannot move the file while it is being accessed (minus kicking your client off the FTP site temporarily). Second, these files are not yet verified and you pose a risk of having a corrupted file.

I know you're excited to get another file ASAP, but I will be adding them in VERY shortly to the other folders.

I don't want to disable downloads from this folder entirely, as I am use text files as 'heads up' messages in there.

Thanks!

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

11 active connections to the FTP site, throwing around 400k/sec of traffic.

eMule throwing around 500k/sec of traffic as well :)

Looking good everyone!

Thank you so much everyone for getting the 3+3 set online. I think at this rate we have that full section online within 5 days! :)

I'm very much wanting to build a chess playing machine. I used to be king on several sites with a Dual MP 2800, but that has got rather outdated. Looking at an 8-way opteron 850/852 machine. I can't wait to watch it be finding mate in 126 from a 6-man file :) LOL

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

149.1GB completed in all, of which 42.1GB is eMule partials. This means there is over 100GB of complete files for download on the FTP thus far! :)

I have around 15GB left to go on the 3+3 files at home. After finishing these, I will stop running eMule for a couple of days and I will dedicate my entire upload to bringing the set online accross the FTP site here, and eMule. :)

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Pushing around about 1300k/sec of traffic very consistantly now :)

Roughly 300k/sec coming in, and 1000k/sec going out! :)

As the line is 10MBPS overall, it will actually be faster for everyone once I have the complete set instead of being shared between the download/upload.

Eventually, I will be limiting it so that there cannot be so many connections per IP, as some are downloading the same file from 4-5 connections. Perfectly fine for now, but I am thinking of restricting this to 2 connections per IP. Of course you can make that 3 connections to the machine by using eMule as well.

Please be adding any files you grab into your eMule shares folder! :)

Any ideas are still more than welcomed. I don't seem to be able to get apache to run correctly. I think this would be a great addition. That way someone could add in the http source to their download. For instance if they have 300MB of a 900MB file complete, they could just download the remaining by adding in a http link to the machine. FTP does not seem to work, at least being passworded.

Apache does not give me a reason why it won't start, but just states there is an error. Checked logs - shows the same. No clue. Any other good HTTP servers someone can suggest? The server is running Windows 2003 64-bit.

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

I think working quite well. 200GB gone out over eMule, and 125GB gone out over FTP.

Whatever the protocol eMule uses - SUCKS. Someone getting 60k/sec from me on eMule can log in and get even 500k/sec+ off of FTP. I hit about 180k/sec off of eMule to the system at home, but FTP gets me 450k/sec (What my ISP capped my internet to after telling me I uploaded too much last month).

205GB completed, moving right along! THANKS MUCH FOR THE FTP UPLOADS TO EVERYONE THAT IS PARTICIPATING! (We have been hitting a pretty consistant 300k/sec coming in!)

I am 2.3GB off from my complete 3+3 set at home, will dedicate my entire upload at home to getting them complete on the server once they are done (i think 12 hours or less now). Uploading right now, but not near max speed. Plus my ISP capped it pretty bad on me.

I am working on ideas of how to get this on a faster line (100mbps perhaps). I am not really finding any cheap options. Perhaps I could just run it one month at this speed, get 32000GB out there in one month if it was hit up at max speed. That would be the complete 10+ sources hitting eMule right there. Hopefully actually 30+ complete sources. :)

I'd be wanting to run 10+ Emule clients in addition to the FTP or something. lol

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Thanks everyone for the continued uploads!

Enjoying my spur of the moment vacation. I see 65GB uploaded over FTP, and I believe at least 50 or so more files from eMule completed.

Probably another 4-5 days until I get back home but will catch up, and organize everything well here.

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

305GB completed :)

Total includes eMule partials and unverified FTP uploads.

Verified another 30GB of completed eMule files (couple copies of uploads in the incoming folder, only lost 400MB by double copies of the such - if everyone just does 3+3/3+3P files there will be no copies at all).

I will also go over the FTP uploads when I get a chance, anything completed/verified from there will be removed from my eMule downloads.

Will also go over this reported bad file, I had been verifying the files then copying them over to the 2nd drive. Obviously need to reverify once they are copied over - not before, slipped my mind. I will reverify everything as well very soon to make 100% sure no other problems.

If you are getting a bad file, please redownload it and then let me know, want to make sure it is 100% good everywhere.

Should be back home in 3-4 days at this point and really have a chance to go over everything. The fiance is telling me "the downloads" are getting priority over her, and I need to get going! :)

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Meh. Didn't mean to delete the last post I made (gone now).

In short, upload to the FTP server is disabled for the time being. Downloads were shortly cut off, but are available again. Quite a few problems in short.

Headin' to get some rest.

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

The FTP site is limited to 10KB/sec per connection, and one connection allowed per IP.

This is largely due to that I'm hitting 1100k/sec up consistantly from eMule, uploading to 53 people at the moment. I think it is best to put the bandwidth into a network to where at the least some is being shared back - and likely much more.

I'm not looking for anyone to be frustrated with the download speeds or otherwise - I am just personally thinking its the best thing for everyone as a whole.

I'd be happy to help out in any way that I can - or if anyone has any specific suggestions I'm all ears. If someone sets up a HTTP site I'd be happy to load a set to it for instance - very easy for me to stop eMule, and even 100GB loaded on a site would take less than 1 days time for me.

I'm looking forward to getting my set completed. I have 776GB completed including the eMule partials. Not so concerned with finishing the set so quickly at this point I suppose, as I am already seeing over 1MB/sec going out over eMule. Maybe a month just getting the rest very easily over eMule.

(Thanks go to jkominek for a large majority of these files he gave to me :) )

Derek
clocks
Posts: 102
Joined: Thu Nov 23, 2006 9:27 am
Sign-up code: 0

Post by clocks »

Updated it to 15KB/sec per connection (Still only one allowed per IP). I think it is still helpful, especially if you have a partial file downloaded from another FTP site and can't finish it there to someone else getting 'your' spot with the connection limits that are allowed.

If you don't see me on eMule (clocks-2), add me as as a source on one of the files, and you should then know me passively in the future, as far as I know eMule saves this. I'm uploading to everyone without queue. IP address (permanent) is 75.126.91.34.

Mostly I am wanting to share on eMule though. I am wanting to share also over HTTP from this machine. That way you could add me as a HTTP source and get files from within your eMule client.

If you you're like me and have a file 99.8% done and lose your connection its just lame LOL - can finish it that way.
User avatar
Pachnes
Posts: 89
Joined: Sun Jun 25, 2006 7:30 pm
Sign-up code: 0
Location: Germany

Post by Pachnes »

Hi Derek,

if you have a nearly complete file and want to finish it soon, just set the priority of this file to "high" and set the priority of the other files, you download, to "low" or "normal".

So you will download this nearly finished file from me, as soon you get your next upload slot from me, as you have a friend slot in my eMule.
Regards,

Thomas
Post Reply