A call to PInvoke function has unbalanced the stack

[baramuse]

Hi everyone,

in our VO app, we were using a C dll for encryption/decryption
The ported code seemto crash when calling the dll.
We do have the source code of the C dll so I tried to see what was bother the Xsharp version of the code :

Code: Select all

#define BF_LONG unsigned long
#define BF_ROUNDS	16

typedef struct bf_key_st
{
	BF_LONG P[BF_ROUNDS+2];
	BF_LONG S[4*256];
} BF_KEY;

typedef BF_KEY	PCRYPT_CTX;

void PCryptInit(PCRYPT_CTX *ctx, unsigned char *key, int count)
{
	BF_set_key(ctx, count, key);
	return;
}

Now on the XSharp side here is the code that was generated by Xporter :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

VOSTRUCT PCRYPT_CTX
    MEMBER DIM P[16+2]	AS DWORD
    MEMBER DIM S[4*256]	AS DWORD

When using that I have a exception :
A call to PInvoke function has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature.

So I tried to match the signature "closer"

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

USING System.Runtime.InteropServices
[StructLayout(LayoutKind.Sequential)];
PUBLIC CLASS PCRYPT_CTX
    PUBLIC DIM P[16+2]	AS DWORD
    PUBLIC DIM S[4*256]	AS DWORD
END CLASS

LOCAL ctx		AS PCRYPT_CTX
LOCAL bKey		AS BYTE[]
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
    
PCryptInit(ctx, bKey, bKey:Length)

Same exception, maybe I'm not looking in the right place ?
Any suggestion ?

[Chris]

Hi Basile,

Can you please show also the exact code you were using to call the function with its original definition?

Also please note that in the modified version of the code, you are passing the first parameter "ctx" (which represents a regular type) uninitialized, so this will definitely not work like that. Using a regular class would also work I guess, but I think it's better to define it as a STRUCTURE PCRYPT_CTX instead. That will not need initialization also.

[baramuse]

Hi Chris,

actually I did init the class, just did not pasted it.
sorry I forget to paste the _DLL invoke code as :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS BYTE[], count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

LOCAL ctx		AS PCRYPT_CTX
LOCAL bKey		AS BYTE[]
ctx := PCRYPT_CTX{}
bKey := BYTE[]{16}{235,89,56,22,81,127,158,80,216,87,210,149,134,206,99,129}
    
PCryptInit(ctx, bKey, bKey:Length)

I tried fist with the VOSTRUCT
Then I tried with a STRUCTURE but can't have fixed lenght arrays ?

Code: Select all

PUBLIC STRUCTURE PCRYPT_CTX
    PUBLIC DIM P[16+2]	AS DWORD
    PUBLIC DIM S[4*256]	AS DWORD
END STRUCTURE

==> Error	XS0573	'PCRYPT_CTX': cannot have instance property or field initializers in structs

[Chris]

Hi Basile,

Ah right, it's not easy to use fixed arrays within STRUCTUREs. It should be possible with the FIXED keyword, but this doesn't seem to work, will further look nto it.

But can you please post the original VO code that you were using and that was calling the original version of the function?

[baramuse]

Sure :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

STRUCTURE PCRYPT_CTX
	MEMBER DIM P[16+2]	AS DWORD
	MEMBER DIM S[4,256]	AS DWORD //That, I don't understand, looks like a 2-dimensional array instead of the 1-dim in the C struct ?

FUNCTION PCrypt(cStr AS STRING, cKey AS STRING, lEncrypt AS LOGIC) AS STRING PASCAL
LOCAL ctx		IS PCRYPT_CTX

PCryptInit(@ctx, String2Psz(cKey), INT(_CAST,SLen(cKey)))
...

[robert]

Basile,
I think you are passing the structure on the stack, but the function expects the address of the structure on the stack.
The C code has this as first parameter:

Code: Select all

PCRYPT_CTX *ctx,

I think the best option is to use the VOSTRUCT.
You are not showing how you call it, but something like this should work.

Code: Select all

LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey), (INT) SLen(cKey))

Alternatively you can declare the variable with 'AS' but then you would have to allocate memory and free that memory and pass the variable without the '@' operator.
Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.
The only thing that I am not sure of is the size of BF_LONG. It is declared as "unsigned liong", but is that a 32 bit or 64 bit variable?
Robert

[baramuse]

Robert,

that's excalty how it was called originally (from VOXPorter) :

Code: Select all

_DLL FUNCTION PCryptInit(ctx AS PCRYPT_CTX, key AS PSZ, count AS INT) AS VOID PASCAL:PgsUti32.PCryptInit

VOSTRUCT PCRYPT_CTX
    MEMBER DIM P[16+2]	AS DWORD
    MEMBER DIM S[4*256]	AS DWORD

LOCAL ctx IS PCRYPT_CTX
PCryptInit(@ctx, String2Psz(cKey),  INT(_CAST,SLen(cKey)))

And was throwing exception.

Finally, make sure that if the C DLL is compiled for x86, that your program also runs in x86.

C DLL was compiled about 15 years ago, so pretty sure it was x86
The XSharp program is targeting x86 as well

[Chris]

Hi Basile,

I suspect it has to do with the 2nd parameter, which expects a pointer to a buffer, but can you please send me (chris at xsharp eu) the dll to give it some tests here? Together with a small VO sample code of using it, the original one, not after VOXporter has maybe made any changes to it.

[robert]

Basile,

Did you try to declare the _DLL FUNCTION "AS VOID STRICT:PgsUti32.PCryptInit" ?

Robert

[baramuse]

Hi Robert, I just tested it, reverted to the ported code only to change PASCAL to STRICT.. and it worked !!
Thanks A LOT !!