2013-10-13

odroid-wheezy-retro 0.8.2 released: kernel 3.8 and updates

With this release I am proud to announce that 3.8.11 is stable enough for our retro-gaming purposes, thus it has been integrated in my retro image.


 
  • now using hardkernel official 3.8.13.10 linux kernel
  • fixed support of Hardkernel upgrade script
  • added MFC firmware (s5p-mfc.fw)
  • supports XBMC (not included)
  • most recent Debian 7.1 packages
  • default partition size is now 2G
  • most recent RetroArch + cores
  • fixed bug with network at 1st boot

2013-09-08

Let's play X-COM on the ODROID U2

One of the classic games I always dreamt to play again (on bigger screens, with huge blocky pixels) is the original X-COM: Enemy Unknown (a.k.a. Ufo Defense).


Not many of them survived every time I played this game


Spoiler: Enemy Unknown works fine on our beloved U2s thanks to OpenXcom :)

I will explain here how to get it running in a few steps. You will need some patience and the original X-COM 1 game. See also OpenXcom's README.

Installing yaml 0.5 packages


You will need slightly more recent yaml packages than those provided with Debian Wheezy.

wget http://ftp.debian.org/debian/pool/main/y/yaml-cpp/libyaml-cpp-dev_0.5.1-1_armhf.deb
wget http://ftp.debian.org/debian/pool/main/y/yaml-cpp/libyaml-cpp0.5_0.5.1-1_armhf.deb

sudo dpkg -i libyaml*.deb

Compiling the sources


git clone https://github.com/SupSuper/OpenXcom.git --depth=0 
cd OpenXcom
mkdir -p buildcd build
cmake ..
CFLAGS="-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard" \
CXXFLAGS="-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard" \
ASFLAGS="-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard" \
make -j5

mv bin/data ~/.local/share/openxcom/data
## customize target of link with your favourite binaries location
ln -s $PWD/bin/openxcom /usr/local/bin/openxcom


At this point, if everything went fine, you have almost finished.


Using the original game


Run the game by providing the original game's directory:

SDL_AUDIODRIVER=alsa /usr/local/bin/openxcom -data /path/to/XcomEnemyUnknown/


NOTE: The prefixed SDL_AUDIODRIVER variable is needed to force usage of ALSA, since I get garbled sound with pulse. If you don't have pulse installed then it's safe to assume that you won't need this environment variable

Game options

From options, select full screen and change resolution to 1280x720 for best gaming experience.

Enjoy it! :)

2013-09-07

Postfix, virtual mailboxes and aliases: priority matters

Sometimes - either for work or leisure - I need to configure Linux servers, and also mail servers.

The '90s lovely logo of Postfix

postfix is a great software that I like to use for mailservers; in this post I want to describe you one feature that works in a counter-intuitive way.

In postfix you can define mailboxes by specifying in your /etc/postfix/main.cf:

virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox_maps

Official virtual MAILBOX example here

You can also specify aliases, e.g. mailboxes that directly forward to other mailboxes, by specifying:

virtual_alias_maps = hash:/etc/postfix/virtual_alias_maps

Official virtual ALIAS example here
Aliases are useful for example when you want to define a catch-all alias forwarding all mail to the postmaster address.

Priority matters!

However, what I dislike is the priority applied to both: aliases have higher priority than mailbox maps (yes, you've read right).

The first time I configured an alias I got all mail delivered to the catch-all address because I thought that - being less catchy - the catch-all addresses would have less priority.

What you need to do is instead add identity definitions to the aliases map, this way:

john@example.com   john@example.com
mary@example.com   mary@example.com
berry@example.com  berry@example.com
@example.com       postmaster@example.com

In above example of virtual_alias_maps, the last line is the catch-all line, while the other lines are inherent to mailbox maps defined in virtual_mailbox_maps.

Now tell me if that's not counter-intuitive, error-prone (because of the duplication) and less easy to mantain.

2013-09-01

The state of hardware acceleration on ODROID U2: 3rd round

There has been fermentation and progress regarding the ongoing 3.8 issues (performance degradation, FIMC/FMC), today I have tested the 3.8 hardkernel linux tree, added some missing patches and could finally complete my benchmarks.

Kernel X11 Driver Resolution glxgears es2gears glmark2-es2 glmark2-es2 --fullscreen
3.0.90 ARM mali 720p ~ 140 fps ~ 599 fps 142 81
3.0.90 ssvb sunxi 720p ~ 144 fps ~ 620 fps 156 91
3.8.13.7 ARM mali 720p ~ 139 fps ~ 659 fps 143 81
3.8.13.7 ssvb sunxi 720p ~ 144 fps ~ 447 fps 87 48

You can compare these tests with the 2nd round of my benchmarks: did you notice the difference? :)




Yes, the performance degradation bug is fixed. Many thanks to ovversun, memeka and mdrjr for all their work and tests!

2013-07-28

Why eBay should disappear tomorrow

I have been using (or better: being affected by...) eBay for almost a decade now. What is my opinion?

It's the worst corporate-blind company in the world, not caring a dime about its customers and enjoying a de facto monopoly as much as they can.
Image courtesy of http://whyebaysucks.info/

This is the list of complaints I have for eBay:
  • crappy user interface that didn't change in the last years, no improvement, no nothing. If the cash cow is milking well, why change anything, right?
  • you cannot change the user interface language and it must be the same language of the nation you have a living address in. Microsoft has the same habit. The message I perceive is: we don't care about you people moving across country borders. How you dare!
  • The HTML editor is...horrible
  • if you save your item as a draft, and then save another item as a draft..*puff!* your previous item is gone. No warning, no nothing.
  • where are our well paid government officials when it's about antitrust? By buying PayPal they created the best monopoly ever, and basically you are forced to use it when using eBay. There are little to zero alternatives, and using others you loose all the "protected customer" rights
  • account verification is a joke. It has a lot of glitches, it keeps looping in a stupid sequence of 3 forms if you haven't yet verified your account from PayPal. So cheap for such a big company, and the problem has been there at least 5 years already!
  • it doesn't matter if your item is conform to description. If the buyer complains, he will get the money back. You are on the weak side by selling on eBay.
  • if you want to sell abroad, eBay will show you that you need 10 feedback comments and 90 days must have elapsed after last sale. But only after you completed the insertion, just to make it more fun for you.
 There are no sound alternatives that I know of, but I seriously hope that they will come to light...this online "service" already smells very bad these days.

2013-07-16

odroid-wheezy-retro 0.8.1 released: ALL RetroArch cores & fixes

A new release!

  • compiled RetroArch with NEON-optimized sound resampler (thanks to Squarepusher)
  • enabled page flipping (DRI2_PAGE_FLIP) for smoother RetroArch experience
  • added vim and missing packages for eduke32 and RetroArch-Phoenix
  • removed modelviewer core, added all RetroArch cores
  • added ~/libretro-super
  • enabled all joystick drivers in kernel

2013-07-14

odroid-wheezy-retro: wishlist & work-in-progress

These days I have been hacking around the ODROID U2 to improve RetroArch performance and better exploit the device's features.

One important achievement is that DRI2_PAGE_FLIP is confirmed to not be negative (at least on 3.0 kernels), but instead gives a positive performance improvement, so please enable it in your xorg.conf configurations.

This is my current wishlist, in priority from most important to lowest:
  • good GPU performance on 3.8 kernel, there has been some progress on this front recently but we are not yet there, 3.0 gives best performance so far Update 1 September 2013: there has been some progress :)
  • G2D support on 3.8 kernel, a support comparable to 3.0 kernel
  • G2D example usage, see also my hello-g2d-odroid and this forum thread
  • G2D-accelerated X11 driver, this could definitively possible once we have a working example
  • MFC-accelerated video decoding, there has recently been some progress on this
  • VSYNC on 3.0 kernel, basically this patch needs to be back-ported so that VSYNC ioctl calls actually do wait for VSYNC - done! Thanks to mdrjr for the patch; it is however verified that VSYNC is not working because of a hardware issue with the HDMI PHY :(
The last one seems easy to accomplish, and the G2D example usage is definitively possible to complete, although nothing else is at my reach :(

G2D acceleration is important to achieve because it would help at scaling anything using the hardware acceleration SoC.

2013-07-11

odroid-wheezy-retro 0.8.0 released: more RetroArch cores

I have been doing some usability testing with Squarepusher and I have applied some improvements that were missing, so here you are with the 0.8.0 cumulative upgrade release.

ODROID Debian Wheezy, retro flavour

Download area

2013-07-10

Windows Update NoNag

Most advanced users, sysadmins and developers that I know are extremely pissed off by the looping nag-screen that Windows Update shows after you install updates.

a bright example of an anti-feature
Although I have found online countless explanations about how to disable the above bastard dialog, they never worked quite well for me.

Since I am a developer, I switched to heavy artillery:

https://github.com/neagix/misc/tree/master/WindowsUpdateNoNag

You can grab the binary directly from here:

https://github.com/neagix/misc/raw/master/WindowsUpdateNoNag/WindowsUpdateNoNag-AnyCpu-1.0-binary.zip

Right after you see that lovely dialog again, run this with Administrator privileges.
NOTE: this tool will disable it only till your next windows update

2013-06-26

ODROID forum members Monthly Award of June

Yesterday I have received news that my project has won the ODROID community Project Monthly Award! Yuppiee!!
One more reason to party :)
Many thanks again to mdrjr and Lisa from Hardkernel for this awesome award - this is really one of the best forum communities I have ever partecipated in. 
I always start trying to make something cool with Gimp, then I end up with some awkward graphics like this. Fair enough, I can't do better anyway :)
This means an extra U2 to play with :) However, I have already 2 and I am not greedy, so I started thinking what it would be the natural usage of such great dev board...

Got any idea?

RetroArch! I quickly decided to have the RetroArch project, in the person of Squarepusher, receive this hardware as a gift for the best of both our growing communities. I see potential for the combination ODROID+RetroArch and on the U2 RetroArch truly shines as a comet of power and great features (like the RGUI).
See how poetic and generous I become when I think about my beloved U2 and retro gaming? :)

More to see on this channel...

My commitment to ODROID community and RetroArch will now be even more motivated. I keep taking care of these releases and news because, as I said in the beginning, I would like to see a skilled development community grow up around these great hardware/software projects.
I have plans for better integrated releases and also tighter testing/development support of RetroArch cores..keep tuned and/or join the efforts if you feel the challenge :)

2013-06-22

Duke Nukem 3D on ODROID

Chocolate Duke Nukem 3D: a sad story


I am a Duke Nukem 3D aficionado and I've recently read with interest Fabien Sanglard's review of Duke Nukem 3D source code. As he writes, the original source code was really horrible.

The Duke on his glorious cover - 3D Realms page here

Fabien also started a Chocolate port, so I grabbed it to try to run the Duke glories on ODROID/Linux.

I realized - in disappointment - that this was a quick hack to have it running on Mac, not a proper chocolate port.
Furthermore the provided Visual Studio solution files were referring to some previous version of the project, with non-matching paths and broken includes - making it also not possible to build it on Windows (this portable is not, my young padawan).

During my own quest to have it building properly on Linux I decided to use Canassa's patches, since they seemed to be done in the reasonable framework of a chocolate port, and then I wrote my own Linux makefiles because Canassa is apparently also a Mac user - thus no Linux portability efforts were added at any time.

My chocolate improvements

In my fork I have added proper CMake support, grab it here: Chocolate Duke Nukem 3D with Linux compile support

However the status of this codebase is almost hopeless:
  • will not work on anything else than 32bit systems
  • some glitch when player dies
  • the 3D Realms logo has some palette quirk, doesn't look like it used to be
Me unhappy :(


If you try to run it on a 64bit system you get only the initial animation, then it goes on "Segmentation fault". That's kind of expected, given the waterfall of integer-to-pointer and pointer-to-integer warnings shown during compilation.

Next step: eduke32


I didn't start from this more advanced port because I wanted to use a faithful version of Duke Nukem 3D, however I am now pleased to see that customization of the built eduke32 is possible to an extent that most advanced features can be turned off.

This is perfect for our ODROID retro-goals! :)

I have done some fixes and prepared a build script, grab them here: eduke32 patch to run on ODROID + optimized build script

Diplomacy is the best feature of this game, as I remember

And enjoy Duke Nukem 3D - it runs flawlessly fast on my U2!

P.S.: I have added the eduke32 binary to release 0.7.3 of odroid-debian-wheezy, or you can download it from here (will only work with Debian Wheezy 7.1 ARM hardfloat)

2013-06-19

odroid-wheezy-base (new image) & odroid-wheezy-retro release update

Time for release! A bugfix release for odroid-wheezy-retro and a new image, even smaller! :)



The new base image, odroid-wheezy-base, contains:
  • debootstrapped Debian root filesystem
  • wired network configured for DHCP
  • SSH
  • configured serial login on UART
  • nothing else!

This is ideal to setup a Debian server with your ODROID, or to kickstart any other customization project you might have. To name only a few: (web)servers, robotics, domotics, top-boxes, you name it!

Release update for odroid-wheezy-retro


The nasty network issue is solved in 0.7.2:



The wildness of the network issue

Some users have reported network issues, however I was never able to reproduce them. At some point I did, and the nature of this issue manifested itself as a very nasty heisenbug.

On first boot the ethernet MAC address was randomly generated and the SSH keys regenerated; it turns out that some randomly generated MAC addresses would make the ethernet never come up, with an error like:

RTNETLINK answers: Cannot assign requested address
Listening on LPF/eth0/xx:xx:xx:xx:xx:xx
Sending on LPF/eth0/xx:xx:xx:xx:xx:xx
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
send_packet: Network is down
receive_packet failed on eth0: Network is down


Correlation is not causation

However, getting to the above conclusion was not easy because last week I observed the following and made wrong correlations:
  1. downscaling issue with one of my customized wheezy-retro SD images (no reason yet verified for this issue, although it is specific that SD image)
  2. network issue with the dedicated wheezy-retro SD image (this was due to the smsc95xx_mac_addr file), letting me for the first time verify the issue of users
  3. found some corrupt (all null bytes) files in the dedicated wheezy-retro SD image
Since file corruption can "taint" everything it touches, I assumed that both the downscaling and network issue were correlated and caused by the first-time observed file corruption.

The lesson is that these issues should instead be considered separate (except 1 and 3 that still need to be verified for correlation), and I have applied mitigation (rsync checksum verification) for (3).

2013-06-09

odroid-wheezy-retro 0.7.1 released: now with desktop icons

0.7.1 released

From the ChangeLog:
  • added idesk as desktop manager
  • added all quicklaunch icons (lxterm, glxgears, es2gears, glmark2-es2, retroarch)
  • other minor improvements/fixes
  • version bump (to 0.7)

Summary

In this version I am glad to introduce a nice & handy addition: desktop icons!

Looks much better now
Due to the constraints of this image (to be lightweight and development-oriented) I did not want to add any proper desktop environment and I fixed & compiled idesk instead. It draws icons. That's all I wanted for this image.

You can launch straight-ahead from desktop the following:
  • LXterminal
  • GLX gears demo
  • ES2 gears demo
  • glmark2-es2 benchmark suite
  • RetroArch (with my patches)

IDesk (fluxbox complement) forked

In odroid-debian-wheezy-retro I am using the great fluxbox as a very lightweight window manager. An useful addition would be icons on the desktop to quick-launch applications.

This is one of the screenshots found at original IDesk SourceForge project. Seriously? >.<

My first choice, fbdesk, is unfortunately broken, so I digged around and the available options for a lightweight desktop were rox-desktop and idesk. rox-desktop comes with a whopping 20 dependencies and idesk is outdated (2005).

The problem with adding so many depencies to a development-oriented distro is that it breaks the philosophy and goals behind it. Instead, by using the most lightweight choice (although less functional) you always leave the user the opportunity to easily make a choice and pick their own, and this is important.

I ended up picking up the orphaned codebase, applying a few patches and polishing it for 2013 :)

Here's my fork: https://github.com/neagix/idesk

I will be mantaining it for a while, so please feel free to use the github facilities to contribute with patches or issue reports.

The state of hardware acceleration on ODROID: 2nd round

Today I have repeated tests with current kernels:

Kernel X11 Driver Resolution glxgears es2gears glmark2-es2 glmark2-es2 --fullscreen
3.0.80 ARM mali 1080p ~ 143 fps ~ 590 fps 137 35
3.0.80 ssvb sunxi 1080p ~ 140 fps ~ 615 fps 151 40
3.0.80 ARM mali 720p ~ 140 fps ~ 600 fps 142 81
3.0.80 ssvb sunxi 720p ~ 144 fps ~ 620 fps 156 92
3.8.13 ARM mali 720p ~ 136 fps ~ 226 fps 61 37
3.8.13 ssvb sunxi 720p ~ 143 fps ~ 226 fps 66 42

Performance seems still halved with 3.8 kernel.
The automated test suite will be released in odroid-wheezy-retro as a script.

2013-06-01

odroid-wheezy-retro 0.0.7 released

0.0.7 released


In this release:
  • kernel 3.0.79 (with disabled fan PWM kern.log verbosity)
  • added xf86-video-sunxifb, you can enable it by copying /etc/X11/Xorg.conf-sunxi to /etc/X11/Xorg.conf
  • added glmark2-es2 binary
  • updated RetroArch binary
  • enabled HDMI 1080p output, (advice: disable it and use instead 720p to use RetroArch)
  • using EXT2 instead of EXT4
  • fixed a few issues related to bootstrapping
ODROID models X/X2/Q are supported by replacing kernel and modules before flashing to SD card, you can get them from mdrjr's builder download area.

Download odroid-wheezy-retro-0.0.7


You can always check the ChangeLog and NextRelease (suggestions are welcome).
Official ODROID forum discussion here.

2013-05-31

The state of hardware acceleration on ODROID (xf86-video-sunxi, xf86-video-mali and kernel 3.8)

Yesterday I tried xf86-video-sunxi (thanks to ssvb, rz2k and xjuan for the support!) and I have to say that this driver bears promise, in particular regarding G2D support, that hopefully in future will bring us 2D hardware acceleration.

The rotating horse is always fun, even when you rotate it.
I have run tests with the following combinations (some old test logs available here):

Kernel X11 Driver Resolution es2gears glmark2-es2 glmark2-es2 --fullscreen glxgears
3.0.79 ARM mali (31 may) 720p ~ 345  fps 160 96 125 fps
3.0.79 ARM mali 720p ~ 345 fps 160 96 125 fps
3.0.79 ssvb sunxi 720p ~ 261 fps 153 89 125 fps
3.8.13 ARM mali (31 may) 720p ~ 226 fps 61 37 ~ 140 fps
3.8.13 ARM mali 720p ~ 226 fps 61 37 ~ 140 fps
3.8.13 ssvb sunxi 720p ~ 226 fps 66 42 ~ 144 fps

Update 8 June 2013: see 2nd round of tests

The compiled sunxi driver and glmark2-es2 are included in version 0.0.7 of the odroid-wheezy-retro SD image.

RetroArch

With both kernels and with both X11 drivers RetroArch performs well in windowed mode (50FPS with rare frame drops with the 3.8 kernel).

Conclusions

  • there is currently a performance issue on kernel 3.8, our beloved mdrjr is working on it
  • glxgears is always software-rendered (see HardwareAcceleration), it's included only for reference
  • 1080p resolution is slow with all configurations, thus I have not targeted it for testing
  • the new Mali drivers (r3p2 released on 31st May) are the same as previous
  • xf86-video-sunxi starts to perform better on kernel 3.8, probably because of the NEON bug that affects 3.0
For now I will keep using kernel 3.0 and ARM mali drivers in odroid-wheezy-retro SD image, however I expect great improvements on both fronts (3.8 kernel and sunxi driver).

2013-05-28

A couple of updates for odroid-wheezy-retro (kernel 3.0.79 and 3.8.13)

I have released 2 scripts to upgrade your odroid-wheezy-retro image to a more recent kernel:


  • upgrade-3.0.75-to-3.0.79.sh, this will upgrade to the most recent 3.0.y kernel. This is currently stable & supported (by me) and will be in next SD image release (0.0.7); also contains a patch to silence the fan message spam in kern.log
  • upgrade-3.0.y-to-3.8.13.sh, this is experimental and subject to change without notice. This script will upgrade to a recent (consider it a nightly build) 3.8 kernel, complete with modules
NOTE: it does not work perfectly with X/X2 e.g. you will have to fix some issues manually
I have not yet decided when to put 3.8 on the odroid-wheezy-retro image - I am going to do some testing before that.

If you are installing 3.8 remember to change your device from /dev/fb6 to /dev/fb1 in /etc/X11/xorg.conf

Official ODROID forum discussion here.

2013-05-27

PCSX-ReARMed on ODROID-U2 (through RetroArch)


These notes are useful to get PCSX-ReARMed (by notaz) running on an ODROID-U2, with full ARM optimizations.

Due to some bug in the current configure/Makefiles, you will not be able to compile&run it succesffully. I got errors like this:
RetroArch [ERROR] :: dylib_load() failed: "pcsx_rearmed_libretro.so: undefined symbol: gteNCLIP_arm".
RetroArch [ERROR] :: Failed to open dynamic library: "pcsx_rearmed_libretro.so"
RetroArch [ERROR] :: Fatal error received in: "load_dynamic()"
And this:
RetroArch [ERROR] :: dylib_load() failed: "pcsx_rearmed_libretro.so: undefined symbol: gteRTPT_neon".
RetroArch [ERROR] :: Failed to open dynamic library: "pcsx_rearmed_libretro.so"
RetroArch [ERROR] :: Fatal error received in: "load_dynamic()"
 
So here you are some instructions to get it running.
First thing, get the libretro port:
git clone https://github.com/libretro/pcsx_rearmed

Apply this patch:

diff --git a/Makefile b/Makefile
index a472492..ed95702 100644
--- a/Makefile
+++ b/Makefile
@@ -10,6 +10,9 @@ CXXFLAGS += $(CFLAGS)
 #DRC_DBG = 1
 #PCNT = 1

+ARCH = arm
+HAVE_NEON=1
+
 all: config.mak target_ plugins_

 ifndef NO_CONFIG_MAK
diff --git a/Makefile.libretro b/Makefile.libretro
index 9436c8a..8a0d67a 100644
--- a/Makefile.libretro
+++ b/Makefile.libretro
@@ -97,8 +97,6 @@ else ifeq ($(platform), arm)
    DRC_CACHE_BASE = 0
    BUILTIN_GPU = neon
    ARCH = arm
-   CFLAGS += -marm -mcpu=cortex-a8 -mfpu=neon
-   ASFLAGS += -mcpu=cortex-a8 -mfpu=neon
 else
    TARGET := $(TARGET_NAME)_retro.dll
    CC = gcc

Now you're (almost) set. Run this script:

export CFLAGS='-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard'
export CXXFLAGS='-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard'
export ASFLAGS='-marm -mcpu=cortex-a9 -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard'
./configure --enable-neon

make -j5 -f Makefile.libretro platform=arm

And enjoy PCSX-ReARMed on your ODROID :)

Issues

It is very slow! It's not performing very well, although all possible optimizations have been enabled. Update 26 June 2013: It's working very fast! Thanks to Squarepusher for the tip of using platform=arm :)

Furthermore, CSO/ISO are not working. Update 26 June 2013: I don't yet know how to work-around this, will publish a new post when I do

2013-05-26

65c816: from assembly to C: a quest for die hard retroengineers!

Some things are impossible to accomplish

For the rest, it's just that you did not try hard enough.

a skeptic (sketchic?) Ray Arnold
This simple fact is behind my interest in 65c816 and the old games that were developed with it: challenge.

Although I had some Intel assembly experience, I clearly overestimated it - deep diving into 6502/65c816 assembly taught me (the hard way) how things work under the hood, and it's a fashioning digital mechanical world.

On reverse engineering

Here be dragons! read more at Wikipedia and here, talking about ethics in computing.

What will happen when there will be a law to regulate everything? You will cease to be free.

This information is provided for learning purposes, not to reverse engineer some company algorithm to steal the precious industrial secrets.
So follow my advice and use reverse engineering for ethical purposes :)

Learning material

A good collection of reference material is essential to succeed. These are the sources I used most often:
In general the other SNES resources at wikibooks are of good level, you can start from there.

As a general advice: DO NOT use badly written manuals/guides (that is the norm nowadays), or anything that leaves you a doubt. Those doubts will be accumulating even if you don't notice it, and you will waste your time in frustration.
When dealing with assembly you really need to reduce number of variables you are dealing with! Make sure you have a grasp of each conditional branching (vs CPU flags) and how ROL,ADC,SBC work - tip: carry is not trivial! ;)

Other interesting pages


What I have learnt:

  • most past games ('90s era) were written directly in assembly (with macro aid, at best)
  • you had to be very creative to fit the limited resources of the consoles
  • dead code and mistakes are commonplace in these games
  • reverse-engineering without tools is hardly possible - I estimate that with double brain size I could probably have a better shot at it

Context limitation

We are human beings, thus we work best when we can focus on a few related abstractions rather than on a wide context with little correlation.

I am going to release a tool, asmdecor, that I developed as an aid for tool-assisted reverse engineering, hopefully they will be of help to other retroengineers as well. 

Labels are your friends!

Doesn't matter if correct - we need labels to reference abstractions. Absolute/Relative memory addresses are not easy to remember.

NOTE: in assembly world there is little difference between a label and a variable, since they both point to a memory location (although in different banks). Here I use them interchangeably for this reason.

I have come up with these rules to assign labels:
  • assign a label to the variables you are currently focusing on
  • remember to change the label once you have a better insight into its usage
  • if you did not succeed, remove the label - next time you will want to start from scratch instead of being conditioned by your previous (inconcludent) ideas
  • if some variable turns out to be too much generically described, drop it
  • make a smart usage of "copyOf", "previous", "counter" etc. basically you should favour relation-rich names
The tool I developed, asmdecor, helps with:
  • variables used only inside a specific procedure, prefixed as "tmp"
  • variables used only for writing, prefixed as "writeOut"
  • variables used only for reading, prefixed as "const"
  • read/write variables, prefixed with "var"
  • procedures and branching point recognition
All variables also come with reference count in their name, to identify "high traffic"/"high value" buffers.

Tracing is your friend!

When trying to understand a piece of code you will find yourself countless times debugging/tracing (step by step) the code and comparing the values with your own version of the algorithm.

The more complex is the piece of code you are trying to understand, the worse it will be - and you will quickly find out that you are doing it ineffectively.

But this was a quest for die hard retroengineers, remember? So I will share with you how I approach this problem - read on!

Vertical tracing


What I have found useful is a technique that I call "vertical tracing" (or "core sampling"), effective on loops and state machines (decompression etc).

This technique consists in creating two indexed stripes, one from your original 65c816 code (master stripe) and one from your reverse-engineered code (carbon copy stripe).
Each of these stripes is made up of multiple (key, value) tuples, created as follows:
  1. normally you want to sample at each iteration, so you will "tether" the loop by using the counter variable as the index of your stripe
  2. you define a "collection spot" for your loop, e.g. a specific variable in your code that you want to monitor in the master stripe
  3. create the corresponding collection spot in your code
  4. compare the master and carbon copy stripes
At step (4) it is important that you throw an exception or otherwise crash when your program is not matching the master stripe.

Example of stripe

1: 0x6300
2: 0x64C0
3: 0x6A00
4: 0x62B0
5: 0x710A
6: 0x712A

In a more compact form:
const char *masterStripe = "0x6300, 0x64C0, 0x6A00, 0x62B0, 0x710A, 0x712A";

By using the compact form you can string-compare it to your carbon copy stripe and thus easily verify behaviour from within your code.

Note that you can also choose to collect the decision taken at a specific branch instead of the value of some variable.

How to generate the master stripe


The most convenient way is to create a trace of the unit of logic you want to reproduce, then use grep/bash scripts to extract the stripe.

Summary: how to write C code from 65c816 assembly

No, there is no magic tool to convert from assembly to C*.

"Isn't tracing a bit like DNA extraction?",
the wise raptor asked.



Tracing, variable context heuristics and procedure identification are the key ingredients.
asmdecor will help you with the latter two, but you still have to analyze it with your mind (and that's the fun part ;) ).

Having advised with all the above, I am left with just a few tips to give you:
  1. identify the procedures, and do not try to study too many procedures at once; ideally you should start with the minimum amount of procedures that are needed
  2. put comments near the branch points with the original addresses, they will be later on helpful for lookup when you are tracing or doing step-by-step debugging
  3. use a original dumps to initialize your buffers, like sections of WRAM and ROM
That's all folks, I hope this read was useful - send me your feedback!




* = It could be an interesting project, however the generated code would be little readable and you still would need to understand it

2013-05-18

odroid-wheezy-retro 0.6 released: you get kernel 3.0.75, 24bit HDMI output and wifi support!

As per title I have just released version 0.0.6 of the image that contains:
  • Hardkernel's linux kernel version 3.0.75, that comes with forced 24bit HDMI output (thanks to libv for the patch)
  • extra wifi drivers :)

If you have kernel 3.0.63 and want to upgrade just use the upgrade script from repository.

I am planning to release an even more streamlined release, so keep tuned! :)
Official ODROID forum discussion here.

2013-05-14

Recipe: compiling BSNES debugger (laevateinn) on Linux

BSNES is the best emulator around, there's little to say about that. Unfortunately the debugger is no more available in current higan because of refactoring and othe changes the author is making.

Let's get the last working bsnes debugger, laevateinn, compiling & working (spoiler: it.does.not.work) on Linux:

git clone git://gitorious.org/bsnes/bsnes.git
cd bsnes
git checkout v089
cd bsnes
nano nall/Makefile # replace "gcc-4.7" with "gcc"
make -j3 target=laevateinn

Finally the debugger will be found in out/laevateinn, but we are not yet there! It can only work with "cartridge folders".

So let's compile purify as well, and run it:

cd ..
cd purify
nano nall/Makefile # here replace "gcc-4.6" with "gcc"
make -j3
./purify output your-rom-path/ target-directory/


Now we are ready to run laevateinn:

cd ..
bsnes/out/laevateinn target-directory/

However, I could not use it because it shows me a non-sexy black screen :(
Possibly you might pick up the task from there.

Meanwhile I am using T.Geiger's snes9x debugger and I have contacted the author about sharing the sources (fingers crossed!)

Update 23 July 2013: instructions for Ubuntu 13.04 provided by user Jay Oster
NOTE: bsnes looks for its system ROMs in ~/.config/laevateinn/

Dependencies

$ sudo apt-get update
$ sudo apt-get install git build-essential libgtk2.0-dev libasound2-dev
# FOR PURIFY
$ sudo apt-get install libx11-dev

Build Laevateinn

$ git clone git://gitorious.org/bsnes/bsnes.git
$ cd bsnes
$ git checkout d418eda9
$ vim bsnes/nall/Makefile # replace "gcc-4.7" with "gcc"
$ make -C bsnes -j 8 target=laevateinn install

Build Purify

$ vim purify/nall/Makefile # replace "gcc-4.6" with "gcc"
$ vim purify/Makefile # add "link += `pkg-config --libs x11`"
$ make -C purify -j 8

PUTRIFY your ROM library

$ ./purify/purify output your-rom-path/ target-directory/

2013-05-11

odroid-wheezy-retro: updated wiki & release

I have started to populate and mantain some wiki pages for odroid-wheezy-retro


ODROID Wheezy Retro Wiki

It's hosted within the Google Code project's wiki, and it fits the purpose quite well so far.

Current release is 0.0.5, I have fixed the nasty issue with Debian automatic network udev rules, thanks to user be.lietaus for reporting it.

Many thanks to mdrjr for providing a backup location for the release files, that you can find here.

Are you an ODROID forum member? Please vote for me on this thread! Well, only if you like this release and want to see more about it coming  :)


2013-05-08

Neagix' excellent (and unofficial) Debian Wheezy Development&Emulation SD image


I have packaged a minimal Debian Wheezy downloadable SD image which achieves the same goals of my previous tutorial (and adds a bit more).
What You See Is What You Will Get 1/3

What You See Is What You Will Get 2/3
What You See Is What You Will Get 3/3

Features of this release

  • very small (187M packed), based on odroid.us base rootfs image
  • using stable kernel 3.0.68 released by Hardkernel
  • X11 hardware accelerated EGL, GLESv1 and GLESv2 (through Hardkernel's proprietary drivers)
  • boots automatically into Xorg 
  • it will not automatically turn off HDMI output when idle (so be careful with your LCDs!)
  • basic development packages already installed (gcc, g++, make, autoconf, automake etc)
  • scripts to fetch and build my fork of retroarch (+ related libretro cores)
  • retroarch (with snes9x-next core) and retroarch-joyconfig ready-to-run binaries included (these versions have my patches)
  • testing scripts for sound and 3D

Coming soon features

  • 3.0.75 kernel (I am testing it before adding to next release, but if you prefer you can upgrade using this script)
  • 24bit HDMI output support (I am not sure about this)
The first time you boot the image it will automatically re-generate SSH keys and assign a random MAC address (see /root/bin/bootstrap.sh for reference), so it might take a few seconds longer.

See project home page and instructions to flash the SD image.



Many thanks to mdrjr and all the other nice IRC people for the help provided since the beginning!

DISCLAIMER: this SD image is not officially supported by Hardkernel, nor I assume any responsability for its use and misuse

Previous posts on this subject:
http://neagix.blogspot.com/2013/04/retroarch-on-odroid-u2-with-hardware.html
http://neagix.blogspot.com/2013/05/retroarch-on-odroid-u2-revisited.html

2013-05-05

RetroArch on ODROID-U2, revisited

After my previous blog post I am considering creating a stand-alone, auto-updating modified Linaro image for RetroArch on ODROID-U2, but don't hold your breath for this now :)


Tip: listen to this great song while reading and hacking your ODROID+RetroArch setup

Update 7 May 2013: I released neagix' excellent (and unofficial) Debian Wheezy for Development&Emulation (with X11 hardware acceleration), get it while it's hot!

Some general advices for people approaching console emulation on ODROID-U2:
  • there is currently no known way to natively exploit the ODROID framebuffer (Exynos 4), thus we have to go through Xorg to run any emulator
  • composing desktop environments are bad for you, so no Unity/Compiz/you-name-it, use instead LXDE or XFCE
  • I have experienced kernel panic/oops/alignment trap issues when using my own compiled version of the kernel, thus I advice to stick to kernel 3.0.63 for now (this has proven to be stable so far) issue was due to PSU
  • remove xscreensaver and other screensaver packages, then modify xorg.conf to not automatically turn off DPMS and other screen blank settings
  • RetroArch runs fine with GL video driver and SDL input
  • be kind with your MCC/SD support, it's not like a regular hard disk and wearout factor is present. Nothing to be scared of, but consider moving write-often directories to external mountpoints
  • create a backup script and run it regularly. You don't know when, but your SD/MMC will one day wake up and tell you "So long, thanks for all the fish..."
  • use nobootwait in your fstab mountpoints, and if you are using NFS mountpoints specify an option for short timeout, like timeo=5
  • disable the long timeouts in /etc/init/failsafe.conf, and feel free to comment lines 34-35
  • if you are using lxdm as login manager, consider enabling autologin by uncommenting autologin=username in /etc/lxdm/lxdm.conf (username shall be your actual username)
  • if you are using LXDE, edit/create ~/.config/lxsession/LXDE/autostart and add a single line @retroarch (retroarch binary must be in your $PATH for this to work), this way once you boot ODROID you will be presented with retroarch
Suggested settings block for  ~/.retroarch.cfg:

video_driver = "gl"
video_fullscreen = "true"
video_windowed_fullscreen = "true"
video_aspect_ratio = "1.33333333333333"
audio_enable = "true"
audio_driver = "alsa"
audio_sync = "true"
input_driver = "sdl"
input_joypad_driver = "sdl"


Please note that only the first definition in the configuration file will be read, so I advise to remove duplicate definitions to reduce confusion.

Some patches for RetroArch

After my unsuccessful attempts at contributing RetroArch I've decided to publish these patches in my retroarch fork, hopefully they will be useful by people affected by my same problems.
Issue: when using RGUI and loading a saved state the button release is passed to the underlying core, causing all weird undesired actions

Fix: wait for all input to be released before starting the core, commit patch here

Issue: rather difficult to troubleshoot input issues might be related to wrong joystick configuration, that is also difficult to detect without a jstest-like program

Fix: enable logging of input events directly in retroarch, commit patch here
Feature: halt the system from RetroArch menu, commit patch here

Leaderitis acutis in open source

Leaderitis acutis

In the last days I wanted to contribute to an open source project with some feedback and bugfix/enhancement patches. Although I fully recognize my non-excellent C++ skills, I found the project to be affected by one of the most common diseases in the FOSS world: leaderitis acutis.

This is a purely fictional disease that I just invented and that I have been thinking about recently (I later found out it's used in american politics jargon, although I don't know how closely matches by what I describe here).

I have always been interested in open source philosophy (and also in free software to a lesser extent), thus I always find it appalling deludent when open source projects do not meet the essential expectations (that is very often for the smaller projects around these days).

Symptoms

  1. Project has very few developers and it has not been exposed to the beneficial strength of the "many eyes"
  2. The most prominent developers use the project to foster all their favourite code smells and anti-patterns (lack of comments, cryptic/mangled code, Baklava code, you-name-it)
  3. Issues, patches and any feedback is negatively considered as a critic or even as an offence (although this might have roots deep into cursed developers' personality)
    Sketch by 3-3 studios
  4. The attitude towards any form of feedback that is not a plain compliment is "Sheesh! This is my little precious, you won't touch it!", missing the point that open source is not your own thing, not even when you start it or are the main contributor
  5. As a last resort solution to any civil and logic argument, developers/mantainers will assert that you have no right to express your opinion, or that your opinion is not supposed to be worth anything in first place. To justify this assertion and enforce the paradigm that they should be trusted as superior, God-gifted owners of truth, they will either:
  • specify that your total contributions amount to 0% and their contributions to 99%, cheerfully living in a chicken-and-egg paradox (or simply barring entry to potential contributors to mantain their sphere of influence untouched)
  • possibly try with any form of personal attack like "you make mistakes, thus your opinions are not trustable".

Cure

There is no known cure for leaderitis acutis, although progress has been verified with personal growth of the involved developers and mantainers that generally happens when time tells them that open source is not about going solo/berserk.