Hello
I uploaded a part of prg. In this part i am reading a DB sequential and building a array. The DB has only 1000 records.
VO needs 0.42 seconds and X# incredible 5.08 seconds
How can i make it faster? I think its the aadd() function.
Horst
X# slow - maybe the array handling ?
X# slow - maybe the array handling ?
- Attachments
-
- part.txt
- (2.91 KiB) Downloaded 77 times
X# slow - maybe the array handling ?
Hi Horst,
There are a lot of things going on in this code, hard to tell what's causing the slowness, without being able to run it. Please provide a compileable and runnable sample, so we can investigate the issue.
There are a lot of things going on in this code, hard to tell what's causing the slowness, without being able to run it. Please provide a compileable and runnable sample, so we can investigate the issue.
Chris Pyrgas
XSharp Development Team
chris(at)xsharp.eu
XSharp Development Team
chris(at)xsharp.eu
X# slow - maybe the array handling ?
Hi Chris
i removed the Aadd () and the app-part has still 4.8 seconds
I will try to make a example app with the dbf.
Horst
i removed the Aadd () and the app-part has still 4.8 seconds
I will try to make a example app with the dbf.
Horst
X# slow - maybe the array handling ?
Hi Horst,
AFAIK Robert is working on a faster read of DBF files.
VO is relative aggressive in caching, and apparently VFP even more.
The first target in the development of the X# RDD was compatibility over speed, but since also speed is an issue, the development team has worked on this also.
As far as I had understand, the X# RDD will be a bit slower than the one from VO because it is written in (slower) managed code, and that is the price for the compatibility with x64/AnyCPU.
But I have to admit that I also have speed issues with the VO RDD specially with more users on networks, and in these cases I'm using the Advantage Database Server.
Wolfgang
AFAIK Robert is working on a faster read of DBF files.
VO is relative aggressive in caching, and apparently VFP even more.
The first target in the development of the X# RDD was compatibility over speed, but since also speed is an issue, the development team has worked on this also.
As far as I had understand, the X# RDD will be a bit slower than the one from VO because it is written in (slower) managed code, and that is the price for the compatibility with x64/AnyCPU.
But I have to admit that I also have speed issues with the VO RDD specially with more users on networks, and in these cases I'm using the Advantage Database Server.
Wolfgang
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
X# slow - maybe the array handling ?
The DBFCDX Rdd is extremely slow as of XSharp Cahors 2.6a and prior versions . See my post from September 15:
https://www.xsharp.eu/forum/public-prod ... difference
The tests included there will a good test case to, hopefully, see if the next version will speed it up.
https://www.xsharp.eu/forum/public-prod ... difference
The tests included there will a good test case to, hopefully, see if the next version will speed it up.
X# slow - maybe the array handling ?
Jamal,
For the next build we have improved the RDD speed in Exclusive mode A LOT.
In one of the builds after that we will try to improve the speed of shared mode as well, but that may cause compatibility problems.
FoxPro seems to simply assume nothing has changed and shows "old data" even when another user has changed the data in the mean time.
It waits until you try to update the data before checking for changes made by other users.
And after a timeout (SET REFRESH TO) it refreshes the local caches of the data.
One of the things that has a negative impact on the speed, even in exclusive mode, is that the data inside DBF files is always Ansi and we have to convert it to Unicode.
A positive side effect of the changes that we did is that we can now also access DBF files larger than 2 Gb.
Unfortunately that may cause problems with shared access since the locking scheme then overlaps with the record contents.
We will add a (configurable) option to switch to Harbour locking scheme for large files when your DBF exceeds the 2 Gb.
Robert
For the next build we have improved the RDD speed in Exclusive mode A LOT.
In one of the builds after that we will try to improve the speed of shared mode as well, but that may cause compatibility problems.
FoxPro seems to simply assume nothing has changed and shows "old data" even when another user has changed the data in the mean time.
It waits until you try to update the data before checking for changes made by other users.
And after a timeout (SET REFRESH TO) it refreshes the local caches of the data.
One of the things that has a negative impact on the speed, even in exclusive mode, is that the data inside DBF files is always Ansi and we have to convert it to Unicode.
A positive side effect of the changes that we did is that we can now also access DBF files larger than 2 Gb.
Unfortunately that may cause problems with shared access since the locking scheme then overlaps with the record contents.
We will add a (configurable) option to switch to Harbour locking scheme for large files when your DBF exceeds the 2 Gb.
Robert
XSharp Development Team
The Netherlands
robert@xsharp.eu
The Netherlands
robert@xsharp.eu
X# slow - maybe the array handling ?
Hi Robert,
that is great to hear, thank you very much!
Regarding the speed in the shared mode: I have noted that also ADS is slow committing changes, so sometimes even in the same program instance you will see old values. Therefore I had to add small delays in some places of my program.
Theoretically a look to the header of the DBF file should be enough so see if a program has changed the data, but it will always be a bit of risk - and only a server engine like a SQL database will be able to solve that (what makes me return to the idea of XDS - the X# Database Server <g>).
Wolfgang
that is great to hear, thank you very much!
Regarding the speed in the shared mode: I have noted that also ADS is slow committing changes, so sometimes even in the same program instance you will see old values. Therefore I had to add small delays in some places of my program.
Theoretically a look to the header of the DBF file should be enough so see if a program has changed the data, but it will always be a bit of risk - and only a server engine like a SQL database will be able to solve that (what makes me return to the idea of XDS - the X# Database Server <g>).
Wolfgang
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
X# slow - maybe the array handling ?
Wolfgang,
I am afraid that a look in the DBF header is not enough:
- The DBF header only changes when a new record is appended.
- The DBF header does not change when a record was updated, unless the last change was on a different day. In that case the LastUpdate field is changed.
- The CDX and NTX header however always get updated when an index key was changed.
However...
There is a 4 byte field in the DBF header (DBASE_LAN, only used by Dbase AFAIK) that we could use for a version counter:
https://github.com/X-Sharp/XSharpPublic ... er.prg#L31
We may be able to increment this field for every update to a DBF, so other stations can see that the file was updated.
That means that this page from the DBF should never be cached but all the other pages could be cached, as long as the version number in this location is unchanged.
I'll see what I can do.
Robert
I am afraid that a look in the DBF header is not enough:
- The DBF header only changes when a new record is appended.
- The DBF header does not change when a record was updated, unless the last change was on a different day. In that case the LastUpdate field is changed.
- The CDX and NTX header however always get updated when an index key was changed.
However...
There is a 4 byte field in the DBF header (DBASE_LAN, only used by Dbase AFAIK) that we could use for a version counter:
https://github.com/X-Sharp/XSharpPublic ... er.prg#L31
We may be able to increment this field for every update to a DBF, so other stations can see that the file was updated.
That means that this page from the DBF should never be cached but all the other pages could be cached, as long as the version number in this location is unchanged.
I'll see what I can do.
Robert
XSharp Development Team
The Netherlands
robert@xsharp.eu
The Netherlands
robert@xsharp.eu
X# slow - maybe the array handling ?
Hi Robert,
that would be great!
Wolfgang
that would be great!
Wolfgang
Wolfgang Riedmann
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
Meran, South Tyrol, Italy
wolfgang@riedmann.it
https://www.riedmann.it - https://docs.xsharp.it
X# slow - maybe the array handling ?
Hello
So i will have passion for the next update. in the meantime i saw some things i can write different to make it faster.
i am making now a standalone exe , so i can test the speed later with the same code and the new x# update.
Horst
BTW the X# team is realy making a good job, and they have alsp passion with one like me
Thanks
So i will have passion for the next update. in the meantime i saw some things i can write different to make it faster.
i am making now a standalone exe , so i can test the speed later with the same code and the new x# update.
Horst
BTW the X# team is realy making a good job, and they have alsp passion with one like me
Thanks