STRCONV(), take #2

[atlopes]

Irwin,

Since you're in the process of increasing VFP functionality in X#, you may consider the proposal of StrConv() at https://www.xsharp.eu/forum/topic?p=18458, which was left behind without being integrated into the X# runtime.

The code appears truncated, but the X# team may have access to the original post's source. If not, I can upload it again.

[Irwin]

Hi Antonio,

Nice to see you here. Yes, you're correct, the code in the old thread is truncated, so please do upload the full source code again here. I'd be delighted to use your work as the foundation for the official implementation.

My plan is to:

1. Integrate your logic into the current XSharp.VFP project (most likely in StringFunctions.prg)
2. Verify and/or Add the neccesary DEFINE constants.
3. Translate your QuickTests examples into standard xUnit test for the repository.

Looking forwar to your code so we can finally cross STRCONV() off the missing list!

Irwin.

[atlopes]

Ok, Irwin, here it is.

Feel free to adjust/rewrite as you will see fit.

Code: Select all

USING System.Text   
USING System.IO
USING System.Globalization
USING XSharp.Core

FUNCTION Start() AS VOID 

    LOCAL TestNr AS Int
    
    TestNr = 1

    // Base64
    QuickTest(TestNr++, 'StrConv("Abcd", STRCNV_SB_BASE64)', StrConv("Abcd", STRCNV_SB_BASE64), "QWJjZA==")
    QuickTest(TestNr++, 'StrConv("QWJjZA==", STRCNV_BASE64_SB)', StrConv("QWJjZA==", STRCNV_BASE64_SB), (Binary)"Abcd")
    QuickTest(TestNr++, 'StrConv(0h010203fdfeff, STRCNV_SB_BASE64)', StrConv(0h010203fdfeff, STRCNV_SB_BASE64), "AQID/f7/")
    QuickTest(TestNr++, 'StrConv("AQID/f7/", STRCNV_BASE64_SB)', StrConv("AQID/f7/", STRCNV_BASE64_SB), 0h010203fdfeff)
    // Hex
    QuickTest(TestNr++, 'StrConv("Abcd", STRCNV_SB_HEX)', StrConv("Abcd", STRCNV_SB_HEX), "41626364")
    QuickTest(TestNr++, 'StrConv("41626364", STRCNV_HEX_SB)', StrConv("41626364", STRCNV_HEX_SB), (Binary)"Abcd")
    QuickTest(TestNr++, 'StrConv(0h010203fdfeff, STRCNV_SB_HEX)', StrConv(0h010203fdfeff, STRCNV_SB_HEX), "010203FDFEFF")
    QuickTest(TestNr++, 'StrConv("010203FDFEFF", STRCNV_HEX_SB)', StrConv("010203fdfeff", STRCNV_HEX_SB), 0h010203fdfeff)
    // Unicode
    QuickTest(TestNr++, 'StrConv(StrConv(StrConv("Привет мир!", STRCNV_UNI_DBCS, 1251, 1), STRCNV_DBCS_SB), STRCNV_SB_HEX)', ;
        StrConv(StrConv(StrConv("Привет мир!", STRCNV_UNI_DBCS, 1251, 1.0), STRCNV_DBCS_SB), STRCNV_SB_HEX), "CFF0E8E2E5F220ECE8F021")
    QuickTest(TestNr++, 'StrConv("Привет мир!", STRCNV_UNI_UTF8)', StrConv("Привет мир!", STRCNV_UNI_UTF8), 0hD09FD180D0B8D0B2D0B5D18220D0BCD0B8D18021)
    QuickTest(TestNr++, 'StrConv(e"Bel\xc3\xa9m do Par\xc3\xa1", STRCNV_UTF8_UNI)', StrConv(e"Bel\xc3\xa9m do Par\xc3\xa1", STRCNV_UTF8_UNI), "Belém do Pará")
    QuickTest(TestNr++, 'StrConv(0hC8, STRCNV_DBCS_UNI, 1253, 1) + StrConv(0hC8, STRCNV_DBCS_UNI, 1256, 1)', ;
        StrConv(0hC8, STRCNV_DBCS_UNI, 1253, 1) + StrConv(0hC8, STRCNV_DBCS_UNI, 1256, 1), 'Θب')
    // Extras
    QuickTest(TestNr++, 'StrConv("Γειά σου Κόσμε!", STRCNV_UNI_SB)', StrConv("Γειά σου Κόσμε!", STRCNV_UNI_SB), 0h9303B503B903AC032000C303BF03C50320009A03CC03C303BC03B5032100)

    WAIT

    RETURN 


FUNCTION QuickTest (Test AS Int, Expression AS String, Result AS USUAL, Expected AS USUAL) AS Void
    
    ? Test, Expression, "->", Result,"vs.", Expected, "(" + IIF(Result == Expected, "Success", "FAIL!") + ")"

ENDFUNC

DEFINE STRCNV_SB_DBCS           = 1
DEFINE STRCNV_DBCS_SB           = 2
DEFINE STRCNV_KATA_HIRA         = 3
DEFINE STRCNV_HIRA_KATA         = 4
DEFINE STRCNV_DBCS_UNI          = 5
DEFINE STRCNV_UNI_DBCS          = 6
DEFINE STRCNV_LOWER             = 7
DEFINE STRCNV_UPPER             = 8
DEFINE STRCNV_DBCS_UTF8         = 9
DEFINE STRCNV_UNI_UTF8          = 10
DEFINE STRCNV_UTF8_DBCS         = 11
DEFINE STRCNV_UTF8_UNI          = 12
DEFINE STRCNV_SB_BASE64         = 13
DEFINE STRCNV_BASE64_SB         = 14
DEFINE STRCNV_SB_HEX            = 15
DEFINE STRCNV_HEX_SB            = 16
DEFINE STRCNV_UNI_SB            = 17
DEFINE STRCNV_UNIBE_SB          = 18

DEFINE STRCNV_ID_LCID           = 0
DEFINE STRCNV_ID_CODEPAGE       = 1
DEFINE STRCNV_ID_CHARSET        = 2

FUNCTION StrConv (Expression AS String, ConversionSetting AS Int, RegionalIdentifier = 0 AS Int, RegionalIDType = STRCNV_ID_LCID AS Int) AS USUAL

    RETURN StrConv_Helper(Expression, (Binary)Expression, ConversionSetting, RegionalIdentifier, RegionalIDType)

END FUNCTION

FUNCTION StrConv (Expression AS Binary, ConversionSetting AS Int, RegionalIdentifier = 0 AS Int, RegionalIDType = STRCNV_ID_LCID AS Int) AS USUAL

    RETURN StrConv_Helper((String)Expression, Expression, ConversionSetting, RegionalIdentifier, RegionalIDType)

END FUNCTION

FUNCTION StrConv (Expression AS USUAL, ConversionSetting AS Int, RegionalIdentifier = 0 AS Int, RegionalIDType = STRCNV_ID_LCID AS Int) AS USUAL

    SWITCH VARTYPE(Expression)
        CASE "C"
            LOCAL StrExpression AS String
        
            StrExpression = Expression
            RETURN StrConv(StrExpression, ConversionSetting, RegionalIdentifier, RegionalIDType)

        CASE "Q"
            LOCAL BinExpression AS Binary
        
            BinExpression = Expression
            RETURN StrConv(BinExpression, ConversionSetting, RegionalIdentifier, RegionalIDType)

        OTHERWISE
            THROW ArgumentException{"Incorrect parameter.", nameof(Expression)}
    END SWITCH

END FUNCTION
 
FUNCTION StrConvStr (Expression AS Binary, RegionalIdentifier = 0 AS Int, RegionalIDType = STRCNV_ID_LCID AS Int) AS String

    VAR enc = StrConv_GetEncoding(RegionalIdentifier, RegionalIDType)
    RETURN Encoding.Unicode.GetString(Encoding.Convert(enc, Encoding.Unicode, Expression))
            
END FUNCTION

STATIC FUNCTION StrConv_helper (Expression AS String, BinaryExpression AS Binary, ConversionSetting AS Int, RegionalIdentifier = 0 AS Int, RegionalIDType = STRCNV_ID_LCID AS Int) AS USUAL
    
    LOCAL ReturnStr AS String
    LOCAL ReturnBin AS Binary
    LOCAL ReturnType AS String

    LOCAL defenc = Encoding.Default AS System.Text.Encoding
    LOCAL utf8 = Encoding.UTF8 AS System.Text.Encoding
    LOCAL unicode = Encoding.Unicode AS System.Text.Encoding
    LOCAL enc AS System.Text.Encoding

    LOCAL Indexer AS Int

    SWITCH (ConversionSetting)

        CASE STRCNV_SB_DBCS
             IF RegionalIDType == STRCNV_ID_LCID
                enc = StrConv_GetEncoding(RegionalIdentifier, STRCNV_ID_LCID)
                ReturnStr = defenc.GetString(Encoding.Convert(defenc, enc, BinaryExpression))
                ReturnType = "S"
            ELSE
                THROW ArgumentException {}
            END IF

        CASE STRCNV_DBCS_SB
             IF RegionalIDType == STRCNV_ID_LCID
                enc = StrConv_GetEncoding(RegionalIdentifier, STRCNV_ID_LCID)
                ReturnBin = Encoding.Convert(defenc, enc, BinaryExpression)
                ReturnType = "B"
            ELSE
                THROW ArgumentException {}
            END IF
                
        CASE STRCNV_KATA_HIRA
            ReturnStr = Expression       // not implemented
            ReturnType = "S"

        CASE STRCNV_HIRA_KATA
            ReturnStr = Expression       // not implemented
            ReturnType = "S"

        CASE STRCNV_DBCS_UNI
            enc = StrConv_GetEncoding(RegionalIdentifier, RegionalIDType)
            ReturnStr = unicode.GetString(Encoding.Convert(enc, unicode, BinaryExpression))
            ReturnType = "S"

        CASE STRCNV_UNI_DBCS
            enc = StrConv_GetEncoding(RegionalIdentifier, RegionalIDType)
            ReturnStr = defenc.GetString(Encoding.Convert(unicode, enc, unicode.GetBytes(Expression)))
            ReturnType = "S"

        CASE STRCNV_LOWER
            IF RegionalIDType == STRCNV_ID_LCID
                ReturnBin = defenc.GetBytes(defenc.GetString(BinaryExpression).ToLower(StrConv_GetCulture(RegionalIdentifier)))
                ReturnType = "B"
            ELSE
                THROW ArgumentException {}
            END IF

        CASE STRCNV_UPPER
             IF RegionalIDType == STRCNV_ID_LCID
                ReturnBin = defenc.GetBytes(defenc.GetString(BinaryExpression).ToUpper(StrConv_GetCulture(RegionalIdentifier)))
                ReturnType = "B"
            ELSE
                THROW ArgumentException {}
            END IF

        CASE STRCNV_DBCS_UTF8
            enc = StrConv_GetEncoding(RegionalIdentifier, RegionalIDType)
            ReturnBin = Encoding.Convert(enc, Encoding.UTF8, BinaryExpression)
            ReturnType = "B"

        CASE STRCNV_UNI_UTF8
            ReturnBin = Encoding.Convert(unicode, utf8, unicode.GetBytes(Expression))
            ReturnType = "B"
            
        CASE STRCNV_UTF8_DBCS
            enc = StrConv_GetEncoding(RegionalIdentifier, RegionalIDType)
            ReturnStr = defenc.GetString(Encoding.Convert(Encoding.UTF8, enc, BinaryExpression))
            ReturnType = "S"

        CASE STRCNV_UTF8_UNI
            ReturnStr = unicode.GetString(Encoding.Convert(Encoding.UTF8, Encoding.Unicode, BinaryExpression))
            ReturnType = "S"
            
        CASE STRCNV_SB_BASE64
            ReturnStr = Convert.ToBase64String(BinaryExpression)
            ReturnType = "S"
            
        CASE STRCNV_BASE64_SB
            ReturnBin = Convert.FromBase64String(defenc.GetString(BinaryExpression))
            ReturnType = "B"

        CASE STRCNV_SB_HEX
            ReturnStr = SUBSTR(BinaryExpression.ToString(), 3)
            ReturnType = "S"

        CASE STRCNV_HEX_SB
            VAR SingleBytes = Byte[]{Expression.Length / 2}
            LOCAL Hex AS Int
            Hex = 0
            FOR Indexer = 1 TO SingleBytes.Length
                SingleBytes[Indexer] = Convert.ToByte(Expression.Substring(Hex, 2), 16)
                Hex += 2
            NEXT
            ReturnBin = SingleBytes
            ReturnType = "B"
            
        CASE STRCNV_UNI_SB
            ReturnBin = unicode.GetBytes(Expression)
            ReturnType = "B"

        CASE STRCNV_UNIBE_SB
            ReturnBin = Encoding.BigEndianUnicode.GetBytes(Expression)
            ReturnType = "B"

        OTHERWISE
            THROW ArgumentException {}
            
    END SWITCH

    SWITCH ReturnType
        CASE "S"
            RETURN ReturnStr
        CASE "B"
            RETURN ReturnBin
        END SWITCH

END FUNCTION

STATIC FUNCTION StrConv_GetEncoding (RegionalIdentifier AS Int, RegionalIDType AS Int) AS Encoding
    
    SWITCH RegionalIDType
        
        CASE STRCNV_ID_LCID
            VAR culture = StrConv_GetCulture(RegionalIdentifier)
            LOCAL CP = culture.TextInfo.ANSICodePage AS Int
 
            RETURN Encoding.GetEncoding(CP)

        CASE STRCNV_ID_CODEPAGE
            RETURN Encoding.GetEncoding(RegionalIdentifier)

        CASE STRCNV_ID_CHARSET
            LOCAL CP = 1252 AS Int
            SWITCH RegionalIdentifier
                CASE 128
                    CP = 932
                CASE 129
                    CP = 949
                CASE 130
                    CP = 1361
                CASE 134
                    CP = 936
                CASE 136
                    CP = 950
                CASE 161
                    CP = 1253
                CASE 162
                    CP = 1254
                CASE 163
                    CP = 1258
                CASE 177
                    CP = 1255
                CASE 178
                    CP = 1256
                CASE 186
                    CP = 1257
                CASE 204
                    CP = 1251
                CASE 222
                    CP = 874
                CASE 238
                    CP = 1250
            END SWITCH
            RETURN Encoding.GetEncoding(CP)
    END SWITCH

    RETURN Encoding.Default

END FUNCTION

STATIC FUNCTION StrConv_GetCulture (LCID AS Int) AS CultureInfo
    
    IF LCID != 0
        RETURN CultureInfo.GetCultureInfo(LCID)
    ELSE
        RETURN CultureInfo.CurrentCulture
    ENDIF
    
END FUNCTION

[Irwin]

Hi Antonio,

I have great news! I have just submitted the PR with the full implementation of STRCONV(), heavily based on your code.

I want to thank you for providing such a complete logic foundation. It saved us a lot of time.

Implementation Notes:
To align your code with the current X# Runtime architecture, I made a few adaptations:

1. Structure: Instead of a separate Strconv_Helper function, I integrated the logic directly into the main function for simplicity, while keeping your GetEncoding/GetCulture logic in an internal static helper class to keep the namespace clean.
2. Typing: Replaced VARTYPE() switches with X# internal type checks (IsString, IsBinary) for better performance.
3. Encoding: As Robert suggested back in 2021, I ensured we use XSharp.RuntimeState.WinCodePage instead of just Encoding.Default to guarantee VFP-compatible ANSI behavior on all systems.
4. Tests: I ported your QuickTest cases to standard xUnit tests in the XSharp.VFP.Tests project.

You can see the final code in the PR here: https://github.com/X-Sharp/XSharpPublic/pull/1770

Thanks again for your contribution!

Irwin.

[atlopes]

Hi, Irwin, great to hear. I'm glad I could be of help.