Find me on GitHub
Friday, November 27, 2015
I haven’t been exactly active on this blog ever since I got a full time job, really. My full time job recently became open source though, so if anyone is interested in what I’m doing, they can follow me on GitHub.
We’re working there on a cross platform runtime and a native compiler for .NET Core. The project is still in the very early stages, but you can already use it to build a C# Hello World command line program and compile it into a native console app for Windows, OSX, or Linux. The console app you build is a fully native program with no .NET dependencies (the only runtime you need is the C runtime!). It’s lean and launches fast. It’s the thing C# developers have been dreaming about for years.
Topics: Personal | Comments Off on Find me on GitHub
Celeron J1900 supports EPT
Friday, August 22, 2014
Quick post. I was shopping for a new home server and was eyeing the new Intel Bay Trail CPUs. The only thing that deterred me from buying it was Intel ARK insisting that Celeron J1900 doesn’t support the Extended Page Table (aka Nested Page Table) feature.
From the past I know that ARK is not exactly the best source of knowledge on EPT (go figure) so I kept searching, but I couldn’t find a confirmation. I found a mention of EPT in someone’s /proc/cpuinfo dump, so I bit the bullet and bought it.
C:\Users\Administrator>Coreinfo.exe -v Coreinfo v3.31 - Dump information on system CPU and memory topology Copyright (C) 2008-2014 Mark Russinovich Sysinternals - www.sysinternals.com Intel(R) Celeron(R) CPU J1900 @ 1.99GHz Intel64 Family 6 Model 55 Stepping 8, GenuineIntel Microcode signature: 00000811 HYPERVISOR - Hypervisor is present VMX * Supports Intel hardware-assisted virtualization EPT * Supports Intel extended page tables (SLAT)
There you have it – both VT-x and EPT are supported by this CPU.
Topics: Uncategorized | Comments Off on Celeron J1900 supports EPT
Bluetooth background tasks in WP 8.1
Saturday, July 5, 2014
This weekend I was working on a small Windows Phone project to scratch one of my itches. I wanted to create an app that would connect to a ELM327 Bluetooth adapter in my car whenever the adapter is within my phone’s reach and do something with the adapter in the background. The Bluetooth adapter exposes itself as a simple RFCOMM COM port.
I thought I should write down the steps to accomplish this (mostly for my own future reference). I chose to do this with a C# WinRT app. It might work with Silverlight, but I’m not sure about that. You need to target Windows Phone 8.1 since most of these APIs are new to Phone 8.1.
Manifest
First things first – the Package.appxmanifest changes. The app needs to declare the RFCOMM serial port capability and the background task:
<Package ...> <Applications> <Application ...> ... <Extensions> <Extension Category="windows.backgroundTasks" EntryPoint="ObdGasHelperBackground.ObdTask"> <BackgroundTasks> <m3:Task Type="rfcommConnection"/> </BackgroundTasks> </Extension> </Extensions> </Application> </Applications> <Capabilities> <m2:DeviceCapability Name="bluetooth.rfcomm"> <m2:Device Id="any"> <m2:Function Type="name:serialPort" /> </m2:Device> </m2:DeviceCapability> </Capabilities> </Package> |
The EntryPoint part specifies the full name of the WinRT class implementing the background tasks (more about it later).
Enumerating RFCOMM devices
To enumerate the RFCOMM serial port services of all paired Bluetooth devices, use this snippet:
foreach (var d in await DeviceInformation.FindAllAsync( RfcommDeviceService.GetDeviceSelector(RfcommServiceId.SerialPort))) { RfcommDeviceService service = await RfcommDeviceService.FromIdAsync(d.Id); } |
Note I said paired – if the device is not paired to the phone yet, it will not show up even if it’s within reach. It might be a good idea to show the user a button that launches the Bluetooth settings system dialog to let them pair new devices. Also note another implication of this – a paired device will show up even if it’s not within reach. This is usually a good thing.
The service object has a few interesting properties that can be shown in the UI to determine what device does it represent. Intellisense is your friend.
Registering a background task
Once we have found a device service to out liking (and maybe let the user pick it in the UI), we can register a background task to fire every time the phone sees the device service being advertised.
We need to start by requesting background access:
if (await BackgroundExecutionManager.RequestAccessAsync() == BackgroundAccessStatus.Denied) { // Uh oh, we can't create background tasks. Maybe the limit was reached or the user declined. } |
With that out of our way, we can create the background task with this code. The local variable “service” is the service we discovered above.
string taskName = service.ConnectionHostName.CanonicalName; // Only register the task if it's not registered yet if (BackgroundTaskRegistration.AllTasks.Values.SingleOrDefault(x => x.Name == taskName) == null) { RfcommConnectionTrigger trigger = new RfcommConnectionTrigger { RemoteHostName = service.ConnectionHostName, }; trigger.OutboundConnection.RemoteServiceId = service.ServiceId; BackgroundTaskBuilder builder = new BackgroundTaskBuilder { Name = taskName, TaskEntryPoint = "ObdGasHelperBackground.ObdTask", }; builder.SetTrigger(trigger); builder.Register(); } |
Again, the TaskEntryPoint is the full name of the WinRT class implementing the background task that we haven’t implemented yet.
The background task
Background tasks have to be hosted in a separate WinMD file. It can’t be in your main executable. Also, you need to create a project reference from your main executable to the background task WinMD even though you technically don’t need that for the app to compile. This part is really important. If you forget it, your app will just force close without any diagnostic information when the task fires. Don’t forget to do this.
The code in the background task is really simple. The OS already opened a socket for us, so all we need to do is read and write into it:
namespace ObdGasHelperBackground { public sealed class ObdTask : IBackgroundTask { public async void Run(IBackgroundTaskInstance taskInstance) { var def = taskInstance.GetDeferral(); var details = taskInstance.TriggerDetails as RfcommConnectionTriggerDetails; DataWriter wr = new DataWriter(details.Socket.OutputStream); DataReader rd = new DataReader(details.Socket.InputStream); // Party on! def.Complete(); } } } |
Topics: Uncategorized | 1 Comment »
Always launch new instances
Tuesday, February 25, 2014
If you have Windows 8.1, try this: hit the Windows key, start typing “not” until you see autocomplete for Notepad and hit Enter. Now type something in Notepad and repeat the same thing. Notice that when you hit Enter to launch Notepad again, it will reactivate the previous running instance of Notepad. If you have multiple instances of Notepad, it will reactivate an arbitrary instance. I don’t know who the hell thought this is a good idea. Same thing essentially happens with any desktop app (including the Command Prompt which is another thing that infuriates me).
On Windows 8 there used to be this undocumented registry setting (DesktopAppsAlwaysLaunchNewInstance) that no longer works for search results in 8.1. So we have a problem to fix. I won’t go much into details, but if you want this fixed, you can do this:
- Install Debugging tools for Windows (free download from MSDN)
- Create a directory on your computer to store symbols in (I use c:\localsymbols)
- Create a shortcut on your desktop to run this:
[path_to_debugging_tools]\ntsd.exe -pn explorer.exe -pv -y SRV*[path_to_local_symbols]*http://msdl.microsoft.com/download/symbols -c "eb Windows_UI_Search!SearchUI::Data::SwitchToApp b8 00 00 00 00 c3; q"(replace the two paths to point wherever you need) - Double click the shortcut and repeat the above experiment. You are welcome.
Topics: Reversing | Comments Off on Always launch new instances
OFFICE 2013 AND ALL CAPS
Tuesday, August 27, 2013
I got recently forced into installing Office 2013 on my computer at work. I was not a big fan of the ALL CAPS menus to begin with (and I’m really happy Visual Studio folks have a regkey to turn that Office plague off in their product), but it really surprised me just how OCD I became over OFFICE. I actually tried avoiding opening Outlook just so I don’t have to deal with EVERYTHING YELLING AT ME ALL THE TIME. The time was right to install Office 2013 at home and put on my reverse engineering hat.
Can I make it lower case again?
I did a cursory search for some of the ribbon text strings in my Office installation and it didn’t take long to find the UTF-16 text strings in msointl.dll. Once I had that, I fired up Outlook under WinDBG and went looking for the spot where msointl.dll gets loaded. I set up a breakpoint on load by typing sxe ld msointl.dll, fired up the process and… breakpoint was not hit.
Well, maybe they load it as a data file. Restart, bp kernelbase!LoadLibraryExW, g,
g, g,…. and bingo. Module loaded.
Once I had the string in the memory, I typed s -su 0x0 L?0xFFFFFFFF "Send / Receive" to find the exact address where the ribbon text was loaded. With an address in hand, it was easy to set up a memory access breakpoint (ba r 1 address) and start debugging.
I will spare you the details of how the text string gets copied and inspected a few times before it’s displayed (and how I had to set up new breakpoints to track the copies as well). It didn’t take long though to find the needle in the haystack – place where the breakpoint was hit with LCMapStringEx on the stack. It turns out Office calls this API with a hardcoded LCMAP_UPPERCASE constant. Locate the push instruction in memory, modify 02 to 00, hit F5. Blam, menu looks as pretty as in Office 2010.
Running Office under WinDBG is not very practical, so I decided I will write a small app for this.
Introducing unOFFIC
I could opt to just patch mso.dll, but that’s hardly practical in the world of software updates and digital signatures. Instead I decided to write a memory patch. This would be really trivial had the code to patch been loaded at the time of process startup. Unfortunately, mso.dll is delay loaded. Hence, the usual trick where you CreateProcess a suspended process, patch memory and resume main thread won’t work. I had to write a small debugger. It is the same amount of fun as it sounds :).
The debugging APIs made things really simple, since they give you a notification when a DLL gets loaded and that’s exactly when we should go and patch that bit in mso.dll.
The last problem was to hook it up so that Office apps always launch under my debugger. I came up with hardly an elegant solution, but works for me – Image File Execution Options. It might break some programs that try to run Office apps programmatically and do interprocess communication afterwards (or wait for them to exit), but that’s a small price to pay for a consistent user experience (the Word previewer in Outlook is affected, for example).
Once I had a way to hook it up at launch, I just decided to go the last mile and write an MSI installer with a complimentary microsite at unoffic.migeel.sk. The link to get the sources is included as well. Enjoy!
Obligatory before and after pics:
After:
Topics: Uncategorized | 2 Comments »
A useful Windows 8 shortcut
Tuesday, March 6, 2012
Now that Windows 8 Consumer Preview is out in public, you might have a hard time finding some of the old features you were using often – like the old Control Panel. Press WinKey + X.
You are welcome!
Topics: Uncategorized | Comments Off on A useful Windows 8 shortcut
AVG Mobilation for Windows Phone
Thursday, September 8, 2011
A new app hit the Windows Phone marketplace today that claims to keep your device safe from malware. I immediately became interested in it because:
- I don’t know of any malware for the Windows Phone.
- Even if there was malware that misuses some kind of hole in the Windows Phone security model, this app wouldn’t be able to catch it because of phone’s application model (unless the app itself uses some kind of security hole).
- After installing it, it claimed it offers real time protection that would suggest it’s capable of running in the background.
I would consider it a joke app, if it didn’t come from a well-known antivirus company. (Spoiler: It actually is a joke app, but the joke is on the antivirus company.)
A look inside
To satisfy my curiosity, I downloaded the XAP file of the app with Marketplace Browser and Downloader for Windows Phone 7 and opened it with Reflector. Surprise, surprise, this app was ported from Android (or at least that’s what *Droid namespace names suggest). Funny how the game has changed and instead of porting antivirus software from a Microsoft operating system to Linux, people started doing it the other way around.
The scanning UI is concentrated in the DroidSecurityPOC.Scan class and gets invoked in the OnNavigatedTo method. The OnNavigatedTo method is actually the first nugget:
protected override void OnNavigatedTo(NavigationEventArgs e) { // uninteresting code removed // do the actual scanning, synchronously (we are scared of threads...) (Application.Current as App).malwareCollection.ScanContainingMedia(this.library); // simulate work in the UI even though the scanning is already completed at this point // people will love this this.StartScan(); } |
The StartScan method looks at the number of files to scan, divides 5 seconds with that number and starts a timer to update the “currently scanned” file name in the UI. Scanning will always take 5+ seconds to complete (closer to 5 seconds if you have few files to scan) and most of the time will be spent waiting for the next timer event to fire. Because all the “scanning” already happened in the ScanContainingMedia method, long time before the UI was first updated.
The scanning algorithm
The DroidSecurityPOC.Data.MalwareCollection class is where the hilarity starts. The ScanContainingMedia method is where all the “scanning” happens. It’s split up in 2 parts: scanning your picture library and scanning your music library. The method doesn’t look at anything else (but that’s not much of a surprise given a marketplace application really cannot access anything else).
At this point, I was still giving the app a chance. Maybe it’s scanning for damaged files that can trigger known exploits in music players or picture viewers. All my hopes disappeared when I looked at the code:
private void ScanContainingMedia(PictureCollection mediaFileCollectiont) { // uninteresting code removed // malwareGroup contains a list of known "malware" // for each picture in the library foreach (Picture picture in mediaFileCollectiont) { // for each known malware (because HashSet is overrated) foreach (string str in malwareGroup.MalwareGroupList) { // compare malware name with current file name (!!!!!!!) // NOTE: We call ToLower() on each string to allocate a new string // and never cache the result. This way the garbage collector will // be busy picking up redundant trash and we can have some fun time // with his daughter. // Also, String.Equals(s1, s2, StringComparison.OrdinalIgnoreCase) // is for pussies. if (str.ToLower() == picture.Name.ToLower()) { // uninteresting code - add malware to a collection of "effected malware" } } } } |
Basically, this code couldn’t be less bothered about the file contents. It only looks at the file name and if it matches the predicate, boom, it’s jailed. No questions asked. Do not pass Go. Do not collect $200.
The list of “dangerous file names” is downloaded from a web service and Rafael Rivera can show you the current “definition file”.
The code also contains an unused method that hints at a future update that will actually look at the file contents, but the method makes me really scared:
public bool ScanEicar(Picture picture) { Stream image = picture.GetImage(); image.Position = 0L; while ((image.Position + 0x44L) <= image.Length) { // the garbage collector still doesn't seem to be busy enough, so // let's allocate an array in a tight loop byte[] buffer = new byte[70]; image.Read(buffer, 0, 0x44); // BLAM! Potentially triple the amount of allocated memory by allocating // a string with the contents of the buffer. Note each character // in a string takes up 2 bytes. // Except Convert.ToString will actually return string "System.Byte[]" // for each and every call. What the author probably wanted // is Encoding.ASCII.GetString(). if (Convert.ToString(buffer).Contains(@"X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*")) { image.Close(); return true; } // Scanning fail: the call to image.Read() already moved the position // by 0x44 bytes. What the author probably wanted to do is // image.Position -= 0x43, but if he did that, the while loop would // run for each byte in the file, allocating about 210 MB from the heap // for a 1 MB file, so the algorithm is probably better off this way. image.Position += 1L; } image.Close(); return false; } |
Everything (including the release date) hints at this being some kind of a summer intern project at AVG (if it’s not, it’s very disturbing). But AVG, c’mon. Interns do all kinds of wonky stuff. You really don’t need to ship all of it…
Topics: .NET, Security | Comments Off on AVG Mobilation for Windows Phone
A look at the Windows Phone JIT compiler
Saturday, July 16, 2011
When optimizing a very hot path in my code, I sometimes find it useful to see what code the compiler is generating for me. Many times I can spot things that can be easily fixed by rearranging code or adding some typecasts.
But getting my hands on the CLR JIT-generated code disassembly on the Windows Phone was not easy. If you think it’s as easy as breaking into the Visual Studio debugger and pressing Ctrl-Alt-D, you’ll be disappointed:
No disassembly available.
Luckily for us, at least the Memory window in Visual Studio still works. Getting our hands on the JITted code will be hard, but not impossible.
Let’s write a method that will be easy to spot in the memory window:
public partial class MainPage : PhoneApplicationPage { // ... public uint Foo() { return 0xDEADBEEF; } } |
Now add a call to this method in PhoneApplicationPage_Loaded and set up a breakpoint after the method call to make sure it’s JITted when the breakpoint is hit. Deploy your project to the emulator and break into the debugger. Now let’s find the method in memory.
Because we can’t use unsafe code on Windows Phone, and the System.Runtime.InteropServices.Marshal class is off limits, we have to turn our hopes to reflection. Luckily for us, the System.Reflection.MethodInfo class contains a field named MethodHandle whose Value points to some kind of internal CLR runtime structure (MethodDesc?). Even though it’s undocumented, we can probably recognize pointers in it and try our luck disassembling memory they point to.
Open the Immediate window in Visual Studio and type:
?typeof(MainPage).GetMethod("Foo").MethodHandle.Value
Executing the above statement in my debugging session gave me 0x0658c910. Looking at that offset in the memory window gave me this:
0x0658C910 e8 a0 b6 03 b8 c5 58 06 0e 00 00 06 01 00 86 00
Following the first pointer to 0x03b6a0e8 (remember, little-endian) will give you this:
0x03B6A0E8 5a 89 55 08 83 c4 d0 89 2c 24 8b ec b9 e8 a0 b6 0x03B6A0F8 03 33 c0 89 45 14 89 4d 0c 89 6e 14 89 45 2c 05 0x03B6A108 00 00 00 00 05 00 00 00 00 ba ef be ad de 89 55 0x03B6A118 2c 05 00 00 00 00 05 00 00 00 00 8b 55 2c 8b 6d
See the string ef be ad de? That has to be our code! Dump the contents of the memory window to a file and save it.
Now fire up your favorite ARM disassembler and load the dumped bytes at offset 0x03B6A0E8. Does it look like trash? It is trash! That’s because the code you are actually looking at is x86, not ARM. How is that possible? The JIT compiler in the Windows Phone emulator produces x86 code. It actually makes sense, because running native code is faster than emulating ARM code. This is probably the reason why the Phone emulator needs hardware virtualization and can’t run under Hyper-V. Most of it runs as i386 code! To see the actual ARM code of the method, you have to dump it from your physical device.
But because we already have the x86 code dumped, let’s have a look at it:
loc_3B6A0E8: ; DATA XREF: seg000:03B6A0F4 pop edx mov [ebp+8], edx add esp, 0FFFFFFD0h mov [esp], ebp mov ebp, esp mov ecx, offset loc_3B6A0E8 xor eax, eax mov [ebp+14h], eax mov [ebp+0Ch], ecx mov [esi+14h], ebp mov [ebp+2Ch], eax add eax, 0 add eax, 0 mov edx, 0DEADBEEFh mov [ebp+2Ch], edx add eax, 0 add eax, 0 mov edx, [ebp+2Ch] mov ebp, [ebp+0] add esp, 34h mov [esi+14h], ebp jmp dword ptr [ebp+8] |
You’ll probably quickly notice 4 things:
- It’s a particularly chatty way of saying
mov edx, 0xDEADBEEF; ret. - The method has a weird prolog and epilog.
- The method doesn’t follow the Intel ABI.
- The method uses a rather big
add eax, 0instruction as anop. Anopwith side effects.
First point can partially be explained by the fact that I was running an unoptimized version (the Debug project configuration), but is closely related to the second, third and fourth point: what we are looking at is code generated by an ARM code generator that was hacked to generate x86 code! The last instruction in the listing is a dead giveaway.
Now let’s look at the code dumped from my actual device (disassembled with standard 32bit ARM instruction encoding):
ADD R9, SP, #0 STR PC, [R9,#0xC] MOV R2, #0 STR R2, [R9,#0x14] STR R9, [R11,#0x14] STR R2, [R9,#0x2C] ANDEQ R0, R0, R0 ANDEQ R0, R0, R0 LDR R1, =0xDEADBEEF STR R1, [R9,#0x2C] ANDEQ R0, R0, R0 ANDEQ R0, R0, R0 LDR R1, [R9,#0x2C] LDR R9, [R9] ADD SP, SP, #0x34 STR R9, [R11,#0x14] LDR LR, [R9,#8] BX LR |
This code feels much more natural than its x86 version. Now let’s look at how the code looks if we enable optimizations (the Release configuration):
STR LR, [R9,#8] SUB SP, SP, #0x2C STR R9, [SP] ADD R9, SP, #0 STR PC, [R9,#0xC] MOV R2, #0 STR R2, [R9,#0x14] STR R9, [R11,#0x14] ANDEQ R0, R0, R0 LDR R1, =0xDEADBEEF LDR R9, [R9] ADD SP, SP, #0x30 STR R9, [R11,#0x14] LDR LR, [R9,#8] BX LR |
You’ll probably notice the code is still not very optimal. As it turns out, the JIT code generator heavily favors code generation speed against code quality. To get most of your CPU cycles, you have to be very careful about how you write your code.
I hope this short post will be useful to you when doing your own Windows Phone .NET code generation investigations. I plan to follow up with some notes on what optimizations you can expect from the Windows Phone CLR code generator that I gathered while optimizing my GameBoy emulator to run on my phone.
Two more useful thing to note: when dumping the method to a file, look at the bytes preceding the method body. Every method has some kind of a header that has (apart from other stuff) 2 pointers in it: pointer to the end of the method body and a pointer to the end of method body including the literal pool. It seems like the header is different depending on whether you deploy a retail or debug configuration.
Many times it’s easier to just dump the whole JIT code heap instead of doing it method by method. After you find the address of your method, just scroll up in the Memory window until you hit uncommited memory region (filled with question marks). Then dump everything starting from there to the end of the heap (a big block of zeroes or question marks).
When it comes to choosing a disassembler, you can try GNU objdump, but if you want something painless, IDA Pro is probably your only option. Get the demo and use this workaround to open raw binaries in it.
Topics: .NET, Performance, Programming | Comments Off on A look at the Windows Phone JIT compiler
.Net back- and frontend for GCC
Sunday, March 7, 2010
I recently stumbled upon a very interesting project – a CIL (.Net assembler) backend and frontend for GCC.
Suppose you have source code in one of the languages supported by GCC (C, C++, Ada, Fortran,…) and you want to use the code from managed code (say, a C# application). A CIL backend would allow you to do just that – compile and link the C code into a .Net assembly so you can link it with C# code.
Now suppose you have a managed .Net assembly and want to compile it to native code for one of the architectures supported by GCC (ARM, amd64, x86, MIPS,… on Linux, Windows, Darwin,…). A GCC frontend allows you to do just that. Take .Net CIL code and compile it to native code.
GCC-CLI
There is a branch in GCC allowing you to do just that. The branch is called GCC-CLI. Even though it’s still very experimental, it already allows you to do a few interesting things. For example: compile C code into managed code using the CIL backend and then use the CIL frontend to compile the managed code to native code.
The backend is quite useful already. I was able to compile my pet C compiler into a fully managed EXE. Not even the Managed C++ compiler from Microsoft can do that! (Managed C++ creates mixed-mode assemblies, which are not fully managed and even if you try to get past this limitation with compiler switches, you get burned by the CRT library.)
The frontend only supports a limited subset of CIL and lacks a runtime library that would do garbage collection and other services (like GCC does with Java – libgcj). So it’s not that useful, yet. But the potential is huge!
A Windows binary specially for the readers of my blog :)
You can try it out yourself. The build instructions are here. You can build it with MSYS and MingW on Windows, but the build process is quite painful. If you don’t want to build it yourself, you can download the Windows binary of GCC with 32bit CIL backend here (7-zip archive – I hope you know what to do with that). Download the archive even if you want to build it yourself. There is a text document with a bunch of notes on how I built it.
Unpack the archive somewhere and run bin\_BUILDTEST.cmd to build a hello world C application. To run the generated _test.exe, you’ll need the assemblies from lib\ to be present in GAC, or in the same folder as the EXE (just copy them over to bin\ and be done with it). Otherwise it will crash. Also, .Net framework is necessary (obviously).
Only C is supported in my build. I was not successful with C++ (it looks like the backend can’t handle exceptions) and didn’t try anything else.
Topics: Uncategorized | Comments Off on .Net back- and frontend for GCC
Geosense for Windows
Tuesday, March 2, 2010
Okay, quick post because this is too awesome to not to write about it. Rafael Rivera from withinwindows.com and Long Zheng from istartedsomething.com have put together a driver for Windows Location API in Windows 7. The driver provides geolocation information for location-aware Windows applications using Google’s Location Services. All it needs is your WiFi turned on (or your IP address, but that’s less precise) and internet connection.
I am really impressed by the accuracy of thins thing (2 locations tested so far and the accuracy was ~30 meters). The best thing: it’s free. Go grab it from geosenseforwindows.com.
Topics: Uncategorized | 1 Comment »
