Page 3 of 4
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 7:55 am
by Chris
Hi Horst,
There have been a lot of improvements and fixes in the RDDs since 2020, and lately we haven't gotten any report about index corruption anymore, but of course something might have still slipped, a bug that we are not aware of. Did you make sure you had the index file properly opened when you edited the dbf from the tool?
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 7:58 am
by Chris
Guys,
About dbf speed issues, there have been many improvements in the past months also in this area, but not all cases are improved. If you would like us to (try to) improve performance in your particular scenario, please send us a full sample showing the speed issue and we'll have a look.
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 8:23 am
by Horst
Hello Chris
I was seeking with the tool, so i think the index was open.
If it will happend again, i try to reproduce the error and send you more informations.
On the files i am working with the x# app, i cant see troubles with the indexes.
Horst
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 10:44 am
by robert
Horst,
If you're accessing the DBF/CDX with an external tool, make sure that the tool uses the same collation settings that your app uses.
Otherwise, corruption of the CDX can occur
Robert
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 10:59 am
by wriedmann
Hi to all,
we have several projects were we are running both X# and VO applications on the same database, and we don't have registered index corruptions the last months.
About performance we cannot say a lot because in the network both VO and X# applications are running much slower than in the past, maybe because Microsoft make some changes in the network stack.
I will do some tests in the weekend.
Wolfgang
P.S. for the people that may interest that: all USB-C and Thunderbolt docking stations (both no name and large producers like HP, Dell and Lenovo) have really slow network interfaces, between 30 and 50% of the speed of either integrated network cards or dedicated USB-C network adapters
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Fri Feb 09, 2024 8:57 pm
by Jamal
Hi Chris,
For an example of the speed issue, please see the following post which is still relevant:
https://www.xsharp.eu/forum/topic?p=17140#p17140
Jamal
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Mon Nov 25, 2024 4:30 pm
by Jamal
Hey Guys,
I am still lurking around!
The speed issue is still in X# 2.21.
It took 22.51 minutes this time because I have a faster SSD on a new PC, but it is a long time. The dbf size after append is 20 MB.
The issue seems to occur when a DBFCDX order is active. When an order is not used, the loop is very fast and took only about 7 seconds.
Code:
Code: Select all
FUNCTION Start() AS VOID STRICT
LOCAL cDBF AS STRING
LOCAL aFields AS ARRAY
LOCAL i AS DWORD
LOCAL nStart, nElapsed AS REAL8
RddSetDefault ( "DBFCDX" )
cDBF := "C:\test\mytest"
FErase ( cDBF + ".cdx" )
aFields := { { "LAST" , "C" , 20 , 0 } }
DbCreate( cDBF , aFields)
DbUseArea( ,,cDBF )
DBCreateOrder("last", "C:\test\mytest.cdx", "last") // comment this out and it will be faster
// wait
nStart := Seconds()
? "start : " + ntrim(nStart)
FOR i := 1 UPTO 1000000
DbAppend()
fieldput(1, asstring(i))
// ? i
NEXT
nElapsed := Seconds() - nStart
? "Elapsed: " + ntrim(nElapsed) + " seconds" + " - Minutes: " + ntrim(nElapsed/60.00)
wait
Console.ReadKey()
Jamal
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Tue Nov 26, 2024 4:49 pm
by Horst
Hi Jamal
I tried it also with your sample. On my machine it took 50 min. !
And its the Index.
When i open the file shared and create the index at the end its 54s . In exclusive mode 19s
You can't make the index after the append ?
Horst
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Tue Nov 26, 2024 5:08 pm
by Jamal
Hi Horst,
Actually, that was a workaround which I am using now, but I wanted to mention that the issue still exists when an index file is active.
Hopefully, sometime in the future a fix is found.
Jamal
Re: DBF Append Speed in X# vs VO - Huge Difference
Posted: Tue Nov 26, 2024 7:01 pm
by Chris
Guys,
The fundamental problem with this seems to be that in X# the time needed to complete the test grows exponentially with the number of records, while in VO it grows linearly. Will log this, maybe something can be changed to the index algorithm in X# to speed it up. Without an index, it grow linearly also in X#, which is good.
But is it a realistic real life scenario needing to add millions (or 100,000s) of records at once with the index activated?