x64 calling convention .c to .asm

x64 calling convention .c to .asm

Post by bmFpcm » Thu, 04 Aug 2005 09:21:02


Hey everybody,

First off, I hope I've selected the correct forum to post this in. I'm
looking for help on the calling convention for an x64 driver written in C
that calls into an Assembly file. The majority of my code is in a .c file
with a few small funtions written in assembly (no I cannot use the available
instrinsics due to 2 lines of code).

For testing purposes, I stole the ioclt sample code from the 3790 DDK (it
is very similar to what I need), and added a call to my assembly. I can
compile both files to .obj, and then link them into a .sys, but when I deploy
the driver, OSR Driver Loader (version 2.2) tells me the driver is invalid
(if I comment out just the call to the assembly function, the driver works
like I expect). I'm pretty sure my problem is either in the way I declare the
extern funtion or in the format of my assembly file.

My compiling commands are:
For my .c file: cl /c test.c --- test.obj
For my .asm: ml64 /c test64.asm --- test64.obj

My linking command is:
link /DLL /VERBOSE /ENTRY:DriverEntry /SUBSYSTEM:WINDOWS
C:\WINDDK\3790\lib\wnet\amd64\msvcrt.lib
C:\WINDDK\3790\lib\wnet\amd64\ntoskrnl.lib test64.obj test.obj /OUT:test.sys

Does anybody here see anything wrong with my compile/link commands?

Does somebody here have information on where I can find a properly formatted
assembly file (preferably one that is called from a .c file and not .cpp),
that will compile with the DDK compiler (ml64) ?

Thanks,
nairb
 
 
 

x64 calling convention .c to .asm

Post by Ivan Brugi » Thu, 04 Aug 2005 11:44:21

There ain't msvcrt.dll in kernel mode, as far as I can remember.
If you have the DDK, try first to build the driver with
the ASM file using the standard rules for `*.asm` files
via the SOURCES file.
When the driver correctly builds and loads, you can then analyze
buildxxx.log
and attempt to replicate the result using other unsupported build mechanism.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.yqcomputer.com/

 
 
 

x64 calling convention .c to .asm

Post by UGF2ZWwgQS » Thu, 04 Aug 2005 20:26:04

have you tried /subsystem:native ?
 
 
 

x64 calling convention .c to .asm

Post by bmFpcm » Fri, 05 Aug 2005 05:59:07

Thanks Ivan and Pavel.

My problem was solved by implementing both your suggestions (I had to remove
msvcrt.lib and use /SUBSYSTEM:NATIVE).

Thanks again,

nairb