Fish tank with Puck the Fish Fish tank with Puck the Fish Water on the floor spelling Fish
 

Hex Fiend

March 28th, 2006

One of my side projects has borne some fruit. Meet Hex Fiend, a new hex editor for Mac OS X. (Hex editors allow you to edit the binary data of a file in hexadecimal or ASCII formats.)

Hex Fiend allows inserting and deleting as well as overwriting data. It supports 100+ GB files with ease. It provides a full undo stack, copy and paste, and other features you’ve come to expect from a Mac app. And it’s very fast, with a surprisingly small memory footprint that doesn’t depend on the size of the files you’re working with.

Hex Fiend was developed as an experiment in huge files. Specifically,

  • How well can the Cocoa NSDocument system be made to work with very large files?
  • How well can the Cocoa text system be extended to work with very large files?
  • How well does Cocoa get along with 64 bit data in general?
  • What are some techniques for representing more data than can fit in memory?

Check it out – it’s free, and it’s a Universal Binary. If you’ve got questions or comments about it or how it works, please leave a comment!

(Incidentally, the Hex Fiend main page was made with iWeb!)

Edit: I’ve discovered/been informed that drag and drop is busted. I will put out an update later tonight to fix this.

Edit 2: Hex Fiend 1.0.1 has been released to fix the drag and drop problems. Please redownload it by clicking on the icon above.

 

The Internet!

π = 3.2860203432

sweetness!

Simply awesome.

Wow that’s really funny. I was just looking for a good hex editor the other day. This looks like it’ll fit the bill perfectly. Nice name, too. :)

– Simone

fds

What a nice surprise! So many basic features requests immediately come to mind though that I wonder if making the source code public is in the plans.

Features such as
turning on overwrite mode so you don’t have to keep count of how many bytes you inserted;
searching for UTF-16 strings;
editing resource forks;
specifying the position – byte amount – by a hexadecimal number in the Move Selection dialogs;
or simply increasing the font size.

[...] « The Avie case New Hex Editor Apple’s ridiculous_fish has been working on Hex Fiend. His screenshots include a complete ‘WTF’ [...]

Ross

Looks nice. Thought you should know the back links on the screenshot pages goes to index.html instead of index.php though :)

Simple feature request — opt-cmd-][{} should move/extend the selection forward/back without dropping the sheet.

Thanks for the tip, Ross; I’ve fixed the broken back-links.

Anonymous

Sweet. Just last week I needed a hex editor and quite frankly, HexEdit sucks 0xBEEF.

The Hex Fiend index page has a title of simply “index”. Doesn’t work so well when you’re perusing your history or want to bookmark it.

The program looks yummy, though. I look forward to 1.0.1.

Will you be releasing the code for this? And/or would you be able to talk about the design an implementation of this? It would be interesting to hear about what you had to do to support such large files.

Aaron Jacobs

Wow, this is exactly what I’ve been waiting for.

John

Excellent program. On my computer, it seems to write statistics to the console log — leftover debugging code perhaps?

This is way neato. Now, for some questions.

How did you extend Cocoa to work with such huge files at such fast speeds? I assume almost none of it is standard AppKit anymore, am I wrong about this? Things like only saving the parts of the file that change — how do you go about this?

Obviously everyone wants source code, but I’d be very happy with general overviews.

stern

I’d like to see options for changing the font size, also the window should be horisontally resizable too. I find viewing files at 32 bytes per row is most convenient.

And while we’re wishing for things, statusbar in hexadecimal? Especially when selecting it’s more informative to know that you’ve selected 0xnnnn bytes than from xxxx to yyyy. And maybe a pony too?

None

Does not work on Mac OS X 10.3.9… Not really good.

Alex

I guess I don’t know too much about development, but I’ve never seen a HEX editor used for anything but cracking other applications. Can some people tell me the practical usage of a HEX editor?

John C. Randolph

Bravo! Nicely done.

Man, that’s fast.

-jcr

Good job. I concur with others here, HexEdit is long in the tooth and needed a serious replacement. And I too would like to see the code to this and start adding features here and there. :-)

Stephen Deken

There are plenty of legit uses for a hex editor — things like manually tweaking input, repairing corrupted binary files (resource and zip files especially), that sort of thing.

It’s a powerful tool, to be sure, but there are plenty of uses that aren’t on the dark side.

Tuomas Rosberg
Altan Orhon

Hi, this is pretty nice and lightweight. I’m probably going to start using it instead of HexEditor ( http://www.ex-cinder.com/hexeditor.html ).

-I’m not sure if the Data Inspector would be better as a toolbar of some sort.

- I think you should put a toolbar with an input box for changing the cursor location too, instead of the “Move Selection Forwards” menu item.
With an input box you could also accept negative values instead of having separate Move Selection Forwards/Backwards items.

- I don’t like the address font.

- Consider a small calculator window / panel so I can translate hex, binary, and decimal values without opening Calculator.app / the terminal!

Altan Orhon

Alex: Hex editors are absolutely necessary for writing software that reads or writes any non-plaintext file formats (like TrueType fonts or a GIF image, as opposed to an HTML document)

Does your project shed light on how to work with large data (> 4GB) in memory (as opposed to on disk) within Cocoa / Objective-C applications? If so, it could be quite helpful to those of us who develop scientific programs.

Hi,

great app, just one criticism: There needs to be a separator or something so one can see where the hex area ends and the ASCII area starts. On a small file that doesn’t even occupy one line, or on an empty file, it’s not obvious at all that the left half of the right pane is actually still part of the Hex view, and not the beginning of the ASCII area.

And to the guy who said a hex editor was for cracking: While it *is* a great help in reverse-engineering a file format (which usually is legal) and similar things, it’s also very useful when you’re debugging a compiler that outputs bytecode or binary files. As such, hex editors have been useful and necessary since the dawn of programming.

Cheers,
– Uli

Darn, forgot something: Do you have any plans for adding a template mechanism? I.e. something like ResEdit’s TMPL editor, just that you can of course use a more Cocoa way of storing a template. Would be nice to have a generic editor for file formats, especially if it supported plugin field types so you can e.g. have a bitmap editor or checksum calculator like in Resorcerer.

Some more thoughts: HexEdit lets one open the resource fork of a file as well. Would be nice to maybe allow doing that. I’d probably go for a “resource” and “data” tab, then someone could copy between the two forks, or convert old-style resource forks into data-fork resfiles etc. I don’t really need HexEdit’s built-in disassembler, so I’m not gonna request that.

Also, Resorcerer’s hex editor had the stuff you display in the “Data Inspector” in the status bar instead, which is a little more convenient. It would simply display all data types that made sense for a selected range of bytes (i.e. no 64-bit display when only 4 are selected, OTOH it also showed 4 bytes as a Quickdraw point etc… Would be nice to be able to view something as a CGPoint or long long that way).

On the hex/ASCII area separation: Apparently this is only an issue with files of 16 bytes or less, as then the first line is white and the second line isn’t colored with alternating row colors. So, if you could just have empty files display the row coloring anyway, that’d be quite sufficient to solve the problem.

Cheers, keep up the good work,
– Uli

Empty AREAS of the window, I mean, not files … sigh.

Alex

I’m sorry if I came off as accusatory earlier. Thanks for the info! It’s nice to know this app will help other developers do their thing.

Johannes Fortmann

Thanks So Much!
A few suggestions:
- the gutter font seems really tiny to me. I’m short-sighted, but I’m not blind, so I’d guess other people would have the same problem.
- a display of offsets in hex would be cool; also the selection length should be displayed additionally to the start and end.
- the inspector could show the values of a sizeof(type) aligned chunk under the cursor if nothing else is selected. (i.e. 4-byte aligned for int, the byte under the cursor for a char)
- my favourite hex editor (hedit for NT 3.51) had a search dialog that displays both hex and ASCII search at the same time. Basically, the search input field is just another hex view.
That makes it possible to easily enter mixtures of readable strings and random bytes (e.g. pstrings).

Cosmetic:
- the hex view apparently displays exactly the right amount to fill the window. That looks funny though if you resize the window to show n+1/2 lines: when scrolling, the new lines suddenly pop into view; the easy fix is to just display one additional line.
- undoing changes made through a drag operation causes the view to jump, even though the changes were entirely in the current view area.
- the data inspector seems to mix small and mini control sizes. At least that’s what I believe; anyway, it looks awkward somehow…

Well, that’s a lot of criticism; please understand that it’s all meant to be constructive. I’ve already thought about implementing a hex editor myself, until you came to the rescue :-)

EDIT: there I was, thinking that a 10 billions were a very large number…

Jim Schimpf

Very nice job. I especially like the Data Inspector. I do embedded work and this is very useful to explore memory dumps and such. The display isn’t quite as bright and clear as HexEdit but the speed is incredible.

Thanks very much.

Jim Schimpf

Did you know that your app is killed by that same image that kills Safari from DrunkenBlog ? I got a copy to play with and just for fun I tried to open it, Hex Fiend dies just after clicking open. But don’t feel too bad Hex Edit does the same thing.

Great job!

Wow! That has to be the first time I’ve seen ‘petabyte’ in an actual dialogue…

Please do share some of what you learned in the process of making Hex Fiend :)

Christopher Fox

Sounds like you’ve gotten so much positive feedback for Hex Fiend, you should consider a Hex Fiend Pro that implements many of these features! You could either charge for it, or release it to Apple and have them add it to the developer tools install.

Awesome app!

Delicious Library is really cool, now i used iskysoft dvd-library for mac beta to manage and backup DVDs legally