The Official Machinae Supremacy Forum

General => General => Topic started by: refic on May 04, 2006, 10:52:22 pm

Title: Majestic-12
Post by: refic on May 04, 2006, 10:52:22 pm
I would like to invite everybody to join Majestic-12 (or MJ12). It is a WWW search engine done using distributed computing (like SETI@Home etc). Basically it goes like this: you register to the project and download a client software which uses your internet connection to "crawl" the web. First it downloads these "URL buckets" which contain 10,000 URLs each, then it goes through them, then packs the information they contain and then send the info back to a central server.

Now on the server side these buckets get indexed etc (I don't remember how the process goes) and the information is made available for the search engine (not in real time though - not yet).

This project is far from ready though - the client is working very fine but on the server side there are still a lot of things that need to be done. One thing I really like in this project is that the developer is very active, I can honestly say that at the moment he has dedicated his life for the project. He visits the project's forum many times a day, answering questions and letting people know what he is doing, and he releases updates like weekly.

So, if you're interested in taking part of this project, it's website can be found at http://www.majestic12.co.uk/ - you'll also find a lot more information about the project from there.

Now if you have any questions, you can ask them here or directly at the project's forum.

--

Like every distributed computing project, this one also has teams.. So if we get enough people to join, perhaps we could set up a MaSu team! :)

--

Please ignore any typos.

--

If you made it this far, thank you for reading this.
Title: Re: Majestic-12
Post by: ПФББЧ on May 04, 2006, 11:18:58 pm
this uses my bandwith, right?
Title: Re: Majestic-12
Post by: refic on May 04, 2006, 11:25:04 pm
Yes. The project developer suggests that people with limited (i.e. monthly transfer limit) connections should not join the project.
Title: Re: Majestic-12
Post by: ПФББЧ on May 04, 2006, 11:30:21 pm
I have a shared net connection, so i shan't be joining. best of luck to you all! :)
Title: Re: Majestic-12
Post by: L'homme magique on May 04, 2006, 11:44:45 pm
I may join. My PC is on like all the time anyway.
Title: Re: Majestic-12
Post by: refic on May 05, 2006, 12:00:30 am
Of course you can fully control how much bandwith the client uses.
Title: Re: Majestic-12
Post by: ПФББЧ on May 05, 2006, 12:59:06 am
someone would bitch somehow.
Title: Re: Majestic-12
Post by: Einkoro on May 05, 2006, 03:41:37 am
What they don't tell you is that they are secretly developing Icarus over this distributed grid of computers.  :o
Title: Re: Majestic-12
Post by: slippy0 on May 05, 2006, 04:59:28 am
i would join, but I already have a similar thing running  :'(
Title: Re: Majestic-12
Post by: chompy on May 08, 2006, 09:29:08 am
I already have F@H going for a while now.... too much effort to remove it and installed the new one.
Title: Re: Majestic-12
Post by: refic on May 08, 2006, 09:44:24 pm
MJ12 doesn't use as much CPU as other distributed computing projects. CPU load goes up only when the client is "archiving" a finished URL bucket, and you also have total control over how much CPU it uses (you can set this from options). During normal crawling CPU usage should be only a few %s.
Title: Re: Majestic-12
Post by: refic on June 05, 2007, 01:23:16 pm
http://www.majestic12.co.uk/forum/viewtopic.php?t=2631

:)
Title: Re: Majestic-12
Post by: Torp v2.0 on June 05, 2007, 10:13:29 pm
Interesting project, but I think I prefer Folding@HOME (http://folding.stanford.edu/).

Not that folding has run on my computer for quite some while now.

How much internet capacity is "needed" for this stuff, by the way?
Title: Re: Majestic-12
Post by: refic on June 05, 2007, 10:19:54 pm
If you mean the client, well, there are people running it on dial-ups.. I've dedicated some 9mbits for the project, may add another 5-6mbits soon.
Title: Re: Majestic-12
Post by: WereVolvo on June 05, 2007, 10:32:36 pm
congratz for getting the server named after you, refic ;D
Title: Re: Majestic-12
Post by: Ant on June 05, 2007, 11:11:33 pm
I only have 64 kb/s upload but unlimited use of my net, would 20kb/s even be helpful? :/
Title: Re: Majestic-12
Post by: refic on June 06, 2007, 12:20:08 am
It's the download speed that matters the most.. and yes, I'm sure that any kind of help is welcome :)

For example: http://www.majestic12.co.uk/projects/dsearch/userinfo.php?id=1157
This user doesn't have much bandwidth but he's been with us for a long time and has actually managed to crawl quite many URLs :)
Title: Re: Majestic-12
Post by: Ant on June 06, 2007, 12:53:39 am
Okay well I got it now, been trying to read through the manual to know as much as I can and this does look quite a load better than the other ones that are out there o:

So who's up for Team MaSu? :D
Title: Re: Majestic-12
Post by: refic on June 06, 2007, 04:38:51 pm
Team MaSu would be nice, but we need more members :-\
Title: Re: Majestic-12
Post by: Fallout on June 06, 2007, 05:22:31 pm
Team MaSu would be nice, but we need more members :-\

I'd be willing to join if my internet was any good. I have 16mb broadband but it's capped at 10mb and cuts off randomly, sometimes fails to stay connected for even 5 minutes.

I'm getting some well-needed exercise though, dashing backwards and forth between the rooms resetting the router. :P