PDA

View Full Version : @TTG. RARing the installer to add checksums?


hansschmucker
08/27/2009, 05:44 pm
EDIT: IF YOU CAN'T UNDERSTAND THIS POST, IGNORE IT. AT LEAST DON'T START INSULTING PEOPLE

I just did a few short tests with WinRAR's self-extracting installer module and the results are encouraging. Especially in light of all the CRC errors we're currently getting all over the support forum.

The WinRAR module is capable of performing these checks during decompression. The drawback of course is that there's yet another installer containing the installer (WinRAR can't replace NSIS), which means another GUI (even if it only displays a message while extracting the setup to the temp directory), a delay until the normal installer launches and of course additional points of failure (like a full temp drive).

On the plus side, the compression actually shaves quite a bit of filesize from the NSIS installer (I guess they're still using deflate). Episode 2 came in 257MB and ended up 167MB.

Will
08/27/2009, 05:49 pm
Just out of curiosity, what does their error messaging look like when it actually does fail the checksum?

hansschmucker
08/27/2009, 05:52 pm
"CRC failed in setup.exe" in the log area and "Some installation files are corrupt. Please download a fresh copy and retry the installation." as a message box.

I haven't looked into the license terms and structure of the installer yet, but I would imagine that it could be modified with a simple PE editor.

Edit: Just looked at it in the hex editor, seems like the strings are stored another way. But if you wanted to use it, you could just ask the developer. I'm sure he'd change the strings for you or tell you how to do it.

quickfics
08/27/2009, 08:25 pm
I ran this through Babelfish, and all the above roughly translate to:

"Eggs, eggs, eggs! The postcards of which the man is speaking cost one dollar per. Can you inform me of where I may urinate? Thank you, but have you any water which is bottled?"

Hopes this helps anybody who does not speak nerd.

GaryCXJk
08/27/2009, 09:17 pm
A very loose translation. I didn't really translate certain things correctly due to the fact that it would only complicate the whole matter.

I just did a few short tests with WinRAR's self-extracting installer module and the results are encouraging. Especially in light of all the CRC errors we're currently getting all over the support forum.I did a few short tests with a feature of a program which makes installation files smaller, and the results are very niiiice.

http://www.multiverseworks.com/garycxjkrandomstuff/borat-high-five.jpg

Especially considering people get these errors about corrupt file sizes and stuff we get all over the support forum.

OR:

A few tests with a WinRAR feature has shown that, aside from a lower download size, it also adds a special check to ensure that you have the complete file downloaded.

The WinRAR module is capable of performing these checks during decompression. The drawback of course is that there's yet another installer containing the installer (WinRAR can't replace NSIS), which means another GUI (even if it only displays a message while extracting the setup to the temp directory), a delay until the normal installer launches and of course additional points of failure (like a full temp drive).

This program is capable of checking the right file sizes after the file has been restored to its original size. The disadvantage of course is that you have to deal with another install program with its own interface, the added delay before the actual install program runs and of course some extra points where things can go wrong (for example, a full hard drive due to the extra files that are being made during installation).

OR:

It can check if the files are correct, with the drawback that you'll have to deal with an extra installer with its own problems.

On the plus side, the compression actually shaves quite a bit of filesize from the NSIS installer (I guess they're still using deflate). Episode 2 came in 257MB and ended up 167MB.

On the plus side, compressing makes the installer smaller. Episode 2 was lowered from 257 MB to 167 MB.

OR:

http://www.multiverseworks.com/garycxjkrandomstuff/ttg_obvious.jpg

hansschmucker
08/28/2009, 04:53 am
I ran this through Babelfish, and all the above roughly translate to:
"Eggs, eggs, eggs! The postcards of which the man is speaking cost one dollar per. Can you inform me of where I may urinate? Thank you, but have you any water which is bottled?"
Hopes this helps anybody who does not speak nerd.

Interesting, "Some installation files are corrupt. Please download a fresh copy and retry the installation." is apparently too complex for you. However I still think that the average user can understand it.

shref
08/28/2009, 04:58 am
I ran this through Babelfish, and all the above roughly translate to:



Hopes this helps anybody who does not speak nerd.

whats wrong with nerds?? i prefer the term "geek" btw =P

hansschmucker
08/28/2009, 05:00 am
I did a few short tests with a feature of a program which makes installation files smaller, and the results are very niiiice.

This may surprise you but compressing compressed files doesn't always make them smaller. If it did, we could compress a file over and over again until it finally is 0 bytes in size. But it doesn't work that way. A once compressed file usually gets BIGGER if you do it too often. Since the NSIS installer is already compressed I didn't expect a big win.

hansschmucker
08/28/2009, 05:06 am
I'd like to say, for the record, that my good upbringing stops me from saying what I would like to say about some of the comments in here.

The public version is: These posts are unhelpful, insulting and unnecessary.

Guybrush Threepwood
08/28/2009, 05:23 am
Forgive me if I don't understand (probably), but if you're suggesting making the downloads .rar files - no. The fact that the majority of people don't have WinRAR installed is one of the reasons not to do that.

hansschmucker
08/28/2009, 05:25 am
Self-extracting RARs. You don't need WinRAR for it, you still get an EXE file that is capable of running anywhere. It's just that the EXE basically consists of a minified WinRAR version and a RAR file.

Jazzy
08/28/2009, 05:26 am
A quick question: would this process work for the Wii as well? While I am a PC user, if they can increase how much they can get on the Wii then they will probably be willing to put more stuff in both versions. I'm guessing it comes down to whether it's compatible with the wii software, and how it's compressed for their system to begin with.

hansschmucker
08/28/2009, 05:30 am
Nope, sorry. On consoles you're stuck with whatever the console maker is providing for installation.

It's really only of use for the PC version as a quick fix for the corrupt download problems. It's not the ideal solution, it's just the easiest one.

Jazzy
08/28/2009, 05:40 am
Shame, though to be expected I suppose.

Cheers for the info and the testing! :)

wisp
08/28/2009, 05:42 am
what is going on here? i mean, hans made a simple suggestion about how to solve a problem and there are indeed some comments that might be considered quite rude. it's not like his idea is complete nonsense or something. i assume, using a rar so compress the installer further and add crc, may be not a real option for telltale, but it's still better than just complaining.

This may surprise you but compressing compressed files doesn't always make them smaller.
right, you wouldn't believe how often people ask me, why their zipped or rared jpg-files are bigger than the uncompressed ones.

GaryCXJk
08/28/2009, 09:27 am
I'd like to say, for the record, that my good upbringing stops me from saying what I would like to say about some of the comments in here.

The public version is: These posts are unhelpful, insulting and unnecessary.For someone who seemingly knows a lot, you sure do miss the edit button.

Also, people don't really apreciate it if people act too smug about their own knowledge. Just because you know a lot doesn't mean others should, and people who don't understand do deserve at least some explanation. If you really wanted this tech talk to reach the Telltale staff you should have just PM'd them directly. They're not the kind to ignore those messages.

hansschmucker
08/28/2009, 09:31 am
I'd love to explain. I just don't like random insults. You would just have had to ask instead of starting to insult me.

This was a followup to a previous talk I had with Will in another thread where it was entirely offtopic and I thought I'd move it to a new thread to not clutter it up anymore. I also PMd Will but I didn't see any benefit doing this in secret. Well, now I do. Thank you so much for making me realize that I should stick to the lowest common denominator and not post anything in the forums that some people may not understand because I will get random insults in return.

Rawr
08/28/2009, 10:05 am
Sucks that people acted like that. I thought it was indeed a good idea. It's not like you need any other programs to open the exes. It would just mean that before you get to the installer, you would have to wait a few seconds for the exe to extract, which is a small price to pay for removing the installer corruption problems. I suppose the other option to remove the corruption problem would be to have a download manager which included a check, however that would be much more difficult to do.

hansschmucker
08/28/2009, 10:07 am
I've asked the mods to delete this thread in the meantime. Apparently we can't discuss this out in the open. But still: thanks, Rawr

BlueToxic
08/28/2009, 10:20 am
Sounds like a good idea, and since the file size is reduced with the RARing maybe they won't have to compress the textures and sound as much as they do now.

GaryCXJk
08/28/2009, 10:24 am
I'd love to explain. I just don't like random insults. You would just have had to ask instead of starting to insult me.

This was a followup to a previous talk I had with Will in another thread where it was entirely offtopic and I thought I'd move it to a new thread to not clutter it up anymore. I also PMd Will but I didn't see any benefit doing this in secret. Well, now I do. Thank you so much for making me realize that I should stick to the lowest common denominator and not post anything in the forums that some people may not understand because I will get random insults in return.

Wait, what, two people posting something moderate you take as offensive? Face it, you and I are geeks, nerds, whatever you call it. Stop being in denial.

shref
08/28/2009, 10:28 am
so hansschmucker do you work for telltale or were you just trying to help them??

hansschmucker
08/28/2009, 10:30 am
Oh, I'm not in denial about it. But there's a difference between the words being used as an insult or as normal language. Besides, it wasn't just those words that set me off. It was this general "hey, we've got no idea what he's talking about. Should we stop and think about it? Or maybe ask? No, let's just rip him to shreds!" thing.

Edit: I'm not working for Telltale... I'm just a guy who wants companies and products that he likes to succeed.

DjNDB
08/28/2009, 10:31 am
I suppose the other option to remove the corruption problem would be to have a download manager which included a check, however that would be much more difficult to do.

Not that difficult. The BitTorrent protocol offers that feature.

hansschmucker
08/28/2009, 10:34 am
But it still needs a client (except if you're using Opera) and that client has a certain ... reputation. (You and I know that P2P is just a technology that can be used for good or bad, but try to explain this to the man on the street)

DjNDB
08/28/2009, 10:40 am
But it still needs a client (except if you're using Opera) and that client has a certain ... reputation. (You and I know that P2P is just a technology that can be used for good or bad, but try to explain this to the man on the street)

True. I just mentioned it out of geek habit.

DjNDB
08/28/2009, 10:46 am
I've asked the mods to delete this thread in the meantime. Apparently we can't discuss this out in the open. But still: thanks, Rawr

I could do that, but that would be sad. How about cleaning it up and moving it to support instead?
It's usually a different atmosphere over there.

If you still want it deleted though just say so, then i'll do that right away.

hansschmucker
08/28/2009, 10:47 am
Well, half an hour ago you wouldn't have had to ask, but now there's actually something worth saving here ... so move it over and clean it up a bit.

hansschmucker
08/28/2009, 11:04 am
The thing about download managers is that they take away control .. I have my system set up with the parameters I like and that are best for my connection. Plus I get a whole lot of info that you wouldn't show to the average user.

A dedicated Telltale Download manager would ignore all that.

Of course an OPTIONAL download manager would be possible, but I'm not sure it would be worth the costs just for an option. WinRAR on the other hand is cheap and solves most issues.

Will
08/28/2009, 11:04 am
Just for the record, I support threads like this. Couple of us in the office were talking about the thread just this morning.

BlueToxic
08/28/2009, 11:12 am
I also think WinRAR is a better solution than a download manager.

But an optional .torrent file would be great for those who know how to use it. But that would probably confuse a lot of people.

shref
08/28/2009, 11:16 am
I also think WinRAR is a better solution than a download manager.

But an optional .torrent file would be great for those who know how to use it. But that would probably confuse a lot of people.

i just wouldnt require a torrent program. if they were to just have the .torrent thatd be fine(and id probably use that one) but dont make it mandatory. the more software you need to install, the more of a hassle it is. especially if you happen to already be low on disk space.

hansschmucker
08/28/2009, 11:21 am
Thanks Will.

I think torrents are a bit dangerous because a lot of people become very, very afraid when they even see a torrent file. Suddenly you get a lot of questions in the forums whether this site is legal.

From a purely technological point of view torrents are fine and almost perfectly suited for this kind of thing, but users sometimes perceive things differently.

I think if we really wanted torrents, we'd need an official TTG torrent client that basically hides that we're using BitTorrent.

GaryCXJk
08/28/2009, 11:32 am
Why not a special Telltale client? An optional one, that is. You can just automatically download the episodes you want, they automatically appear in your client, or you could just wait until you have a better moment or enough disc space. If it doesn't download correctly, it tries to download the specific block that didn't download right. Perhaps even link it to other features on the site, like the Treasure Hunt. Serial keys or activations would go trough this client, so that you wouldn't always have to enter your username and password or your serial key everytime you download a new episode, regardless what series each episode is.

It would add to the appeal, and it doesn't even have to be all that extensive as STEAM. The games still work independent from the client, so you could at all time close the client and still play the games.

However, I do forsee that it does take a fair bit more server load, as well as the development cost for such client.

Also:

http://www.telltalegames.com/forums/images/buttons/edit.gif

hansschmucker
08/28/2009, 11:34 am
I think the development costs would be the main problem here. It's certainly possible, but it takes work and money ... and I fear those costs outweigh the benefits.

Bandwidth-wise it would very likely be actually be an improvement as you can integrate whatever technology you like in order to optimize the transfer (P2P, compression, ???).

DjNDB
08/28/2009, 11:35 am
But an optional .torrent file would be great for those who know how to use it. But that would probably confuse a lot of people.

I would use the .torrent too.

It usually results in fast downloads, which would be nice especially shortly after a new Episode release.

It would also reduce TTGs traffic. Assuming traffic dictates the filesize that could be another way to make the "...compressed petition" people happier.

hansschmucker
08/28/2009, 11:38 am
Also:

http://www.telltalegames.com/forums/images/buttons/edit.gif

No, you will never get me to use that button for anything but fixing mistakes. I actually like threads lighting up when new content is added.

shref
08/28/2009, 11:38 am
I would use the .torrent too.

It usually results in fast downloads, which would be nice especially shortly after a new Episode release.

It would also reduce TTGs traffic. Assuming traffic dictates the filesize that could be another way to make the "...compressed petition" people happier.

i just wonder how many people would use this and if it would be worth the cost to telltale. of course i could be wrong here

DjNDB
08/28/2009, 11:39 am
I think the development costs would be the main problem here. It's certainly possible, but it takes work and money ... and I fear those costs outweigh the benefits.

Absolutely. The main point is getting the game to the customer. That feature already exists, but could be technologically improved.

A telltale downloader would be a bit overkill to achieve the same goal considering how seldom you would use it.

hansschmucker
08/28/2009, 11:40 am
i just wonder how many people would use this and if it would be worth the cost to telltale. of course i could be wrong here
The costs for a torrent tracker are minimal... it's really not much of a deal. Rent a virtual machine somewhere with enough bandwidth and you're pretty much covered. Installation and maintenance are very easy.

DjNDB
08/28/2009, 11:42 am
i just wonder how many people would use this and if it would be worth the cost to telltale. of course i could be wrong here

Me too. Acceptance is a big '?'.

BlueToxic
08/28/2009, 11:50 am
How about a poll?

DjNDB
08/28/2009, 11:52 am
How about a poll?

I don't think that's a good idea unless it is really intended to use BitTorrent. It would only confuse people.

Rawr
08/28/2009, 11:55 am
Not really very accurate as a lot of people don't go on this forum. =/ I think having an optional .torrent option would be a good idea though. Although that said, I do wonder how many people would use it, I mean the average PC user won't even know what a torrent is. Also sometimes it's faster to use DD than P2P, as I've found out in other places.

hansschmucker
08/28/2009, 12:00 pm
The bigger fear I have is that the average user thinks he knows perfectly what a torrent is: "Some illegal doohickey used by internet criminals to spread viruses", so "anybody who uses torrents is a criminal, QED".

Rawr
08/28/2009, 12:04 pm
That is another problem too, yeah. The ISPs and the Media doesn't help that either. Constantly releasing stories how ISPs have banned such and such an amount of people today after they had used P2P also saying how wrong using P2P is without realising not all P2P is illegal.

hansschmucker
08/28/2009, 12:09 pm
And torrents are the technology that most media coverage currently focuses on.

Really, if you want to use BitTorrent today for anything but downloads targeted at anybody but computer-litigate users (for example for Linux ISOs) you need to hide it. The recent Metal Gear Solid game comes to mind which uses a BitTorrent client to download updates on PS3. There's a very short notice when you initiate the update, but from that point forward they're careful to never use the word BitTorrent ever again.

BlueToxic
08/28/2009, 12:15 pm
It's a shame it has to be like that.

I guess it's better to use WinRAR here because everyone would benefit from that. The only negative thing I can find with that is that you need a little more disk space.

hansschmucker
08/28/2009, 12:18 pm
170MB extra... I think we can live with that. Even if it is decided against using them for the default setup, maybe we can at least keep a copy of each game somewhere on the TTG server so we can direct people there in the event of trouble.

Rawr
08/28/2009, 12:21 pm
Yep, although it would be rather hard to hide it if TT offered a .torrent file. Unless they made it into a P2P Download manager, for example like what Blizzard has.

(Was aimed at post #47) XD

Yeah but the winrar idea does sound best, and is the most easiest to do.

BlueToxic
08/28/2009, 12:25 pm
Yeah, and the game could be less compressed because WinRAR will compress it a little bit.

hansschmucker
08/28/2009, 12:41 pm
Maybe ;)

DjNDB, about the split installer: Sadly WinRAR doesn't tell you exactly which file is corrupt in a split archive. The output if one file is corrupt looks like this:

Extracting setup.exe
Extracting from SiegeOfSpinnerCay.r00
Extracting from SiegeOfSpinnerCay.r01
Extracting from SiegeOfSpinnerCay.r02
Packed data CRC failed in setup.exe
Extracting from SiegeOfSpinnerCay.r03
CRC failed in setup.exe

The last file mentioned above "Packed data CRC failed" is the one that's to blame. In this case r02

DjNDB
08/28/2009, 12:49 pm
The last file mentioned above "Packed data CRC failed" is the one that's to blame. In this case r02

Thanks for letting me know.

quickfics
08/28/2009, 01:23 pm
Interesting, "Some installation files are corrupt. Please download a fresh copy and retry the installation." is apparently too complex for you. However I still think that the average user can understand it.

No offense, insult, or rudeness intended on my behalf, though I do apologize. My humor often does not translate well. (I find myself hilarious, so I can take some comfort in that.)

hansschmucker
08/28/2009, 01:51 pm
No offense, insult, or rudeness intended on my behalf, though I do apologize. My humor often does not translate well. (I find myself hilarious, so I can take some comfort in that.)

Well, I can't say "none taken" as I obviously was offended by the combination of your and GaryCXJk's posts, which I perceived as an unprovoked attack. But I'm willing to forgive and forget. Apology accepted.

SHODANFreeman
08/28/2009, 03:44 pm
Blizzard's downloader for WoW is P2P, maybe do it like that?

hansschmucker
08/28/2009, 05:21 pm
Never seen Blizzard's downloader... is there anything special about it or do you simply mean it as a good example of a downloader that uses P2P under the hood?

Wapcaplet
08/28/2009, 06:57 pm
Forgive my lack of knowledge on the subject, but doesn't NSIS have an MD5 checksum function available through this plugin (http://nsis.sourceforge.net/MD5_plugin)?

Edit: Never mind -- saw Will's post in the other thread saying they had some problems with it.

Rawr
08/28/2009, 07:25 pm
Never seen Blizzard's downloader... is there anything special about it or do you simply mean it as a good example of a downloader that uses P2P under the hood?
I mentioned it above too. It's a good example of using P2P behind the scenes, aye. Basically it's a download manager that uses P2P to get their files to it's users. It has it's fair share of problems mind you like any place, for example when a patch is released, you can be waiting a day just to download something like a 50mb file.

hansschmucker
08/28/2009, 07:28 pm
Got it, but I'm not going to install WoW just to see it :)

I've sent a mail to the WinRAR developer asking him about modifying the UI for the installer, but I guess we won't get an answer until Monday...

hansschmucker
08/29/2009, 04:23 am
Just got an answer from the WinRAR developer:

"Hello,

Sorry, we do not create custom interfaces for SFX modules.
But it is perfectly legal and not very difficult for them to
use a resource editor like Resource Hacker to modify resources
in default.sfx:
http://www.angusj.com/resourcehacker/
They will need to rename default.sfx to default.exe before editing
and return its original name back after that. They can change
dialogs layout, hide and resize dialog elements, edit error messages
with it.

Eugene"

hansschmucker
08/29/2009, 06:39 am
Asche auf mein Haupt. I shouldn't have jumped to the conclusion that just because I couldn't find the string table, it wasn't there. Still, at least we know that he's fine with modifications to the SFX module.

I've made a minimalistic dialog using reshacker and turned it into an SFX module. The file is truncated to save bandwidth, so it doesn't really work, but you can see the dialog.

http://www.tapper-ware.net/files/Telltale/RARSetup-test.exe

BlueToxic
08/29/2009, 08:58 am
Is there any way to make it extract automatically and then remove the files after the setup is complete?

I think some users could be confused by having to choose install location twice.

hansschmucker
08/29/2009, 10:19 am
It does exactly that :) . Extract to temp, run, wait to close, remove temp files.

BlueToxic
08/29/2009, 10:29 am
Ok, I must have done something wrong.

Then I don't see why TTG shouldn't use this.

hansschmucker
08/29/2009, 10:45 am
Hmm, is this what you tried:
http://www.tapper-ware.net/files/Telltale/RARSetup-test.exe

It should show the progressbar, then quit (there's only a truncated and non-working version of the ToMI installer included to give the extraction process something to do)

If you're trying to do it yourself, Advanced/SFX Options/Modes/Unpack to Temporary folder + add filename of setup file under Advanced/SFX Options/General/Run after extraction

BlueToxic
08/29/2009, 11:04 am
Thanks, I never looked at the Advanced settings, I just checked the "Create SFX Archive" box.

Your file instantly give me an error message saying "Windows cannot find 'setup.exe'. Is that how it's supposed to do?

hansschmucker
08/29/2009, 11:06 am
Funny, that's what I expected it to do that, but I don't get the error message. But basically yes: You should briefly see the extraction progress dialog, then it dies.

BlueToxic
08/29/2009, 11:29 am
Maybe they can use silent mode and only change the icon.

hansschmucker
08/29/2009, 11:31 am
No feedback at all while extracting 400MB is a bit much don't you think? But we can modify the UI anyway, so no problem there. The only question that remains for me is how to sign an SFX archive: I never had to work with signed EXEs.

BlueToxic
08/29/2009, 11:45 am
Yeah you are right, it's probably better if you see the progress, and modifying the UI wasn't very hard.

BlueToxic
09/02/2009, 07:42 am
Any luck with signing the SFX archive?