1

Re: SQL server questions

1) Are there plans to allow R6 users/administrations to change/choose the backend via R6 itself? i.e. choose a different SQL server or MSDE instance than R6 was connected to at time of installation?

2) If SQL server (express) is used for a network install of R6 will it run any faster than the old R2k2 did using an Ms Access backend over a network? i.e. will queries be run on the network SQL server and only the answer spit out to the local PCs R6 frontend? or will R6 drag all the data over the network and query it on the local PC before providing an answer (like Access would)?

3) are replacing the missing dictionaries the only major change to R6.8 other than bug fixes?

Cyprian

CyprianPayne
CCW

2

Re: SQL server questions

Hi Cyprian,

I'm not sure if the ability to select alternative SQL Server databases via the front end is a general requirement but if it is then this could certainly be considered.  As to the other questions, SQL Server, MSDE and SQL Server Express/2005 all work the same way, that is, they indeed process the query on the server and return just the results set to the client.  As you say, this contrasts with Access where the entire data is returned to the client machine for processing.

In my experience, Access can perform quite well in comparison with SQL Server but only when there is only a small amount of data, with the data on the same machine as the client application (as in a Recorder Standalone install) and when there are very few concurrent users (i.e. rarely more than 1).  As soon as you move the data onto a network, increase the volume of data, or increase the number of users the performance advantages of client/server become very apparent.  Furthermore, SQL Server has a much 'richer' server programming language allowing complex code to be run on the server itself, further improving performance.  A caveat to this is that when considering a Recorder 6 (or any other SQL Server installation) it is important to put the database on a reasonably quick machine.  The client machine isn't so important as it doesn't process the queries.

Finally, apart from dictionary updates, the Recorder 6.8 upgrade includes the following:
1) Ability for JNCC to issue dictionary updates separately to the application updates.
2) Making the standalone and network install kits compatible with SQL Server Express/2005 (only if you have an existing instance).
3) Enhancements to the way the Import Wizard imports locations and determinations
4) Some nice new facilities for exporting the results set of a report to a variety of formats.
5) Significant performance enhancements to the Taxon Dictionary.

Best Wishes,

John van Breda
Biodiverse IT

3

Re: SQL server questions

I'd add to this that SQL Server scales much better. With Access, it didn't matter how much memory the server or client machines had - it always ran at about the same pace. With SQL Server, the server makes good use of extra memory (we have 3GB in our server) which yields really nice performance improvements.

Regarding the switching of databases, our rare species co-ordinator works offsite most of the time and so has a copy of our database (for read-only purposes) on his laptop along with his own personal database. He can easily switch between the two by changing a registry setting. I've setup two .reg files for him, one which sets his own database in the registry and another that sets ours. It took a bit of setting up on my part (I had to rename one of the databases, for instance) but the end result is very simple and effective.

Charles

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

4

Re: SQL server questions

An addition to the addition - if you switch databases in this way, you should also switch your Object Sheet Path and Map File Path registry settings so that the folder used is unique for each database.  Otherwise you'll have to keep resetting your maps when you switch.

John van Breda
Biodiverse IT

5

Re: SQL server questions

thanks for the replies, very useful.
one more queston about the TITAN middleware (substitute for BDE) that was used in R2k, is that still used in R6? if so, does it have any effect on the speed or otherwise of MSDE/SQL server?

CyprianPayne
CCW

6

Re: SQL server questions

No, its not used anymore in Recorder 6.  For SQL Server access, it uses ADO directly and for Access (only used in the import process) it also uses ADO and Jet.  However, there are still parts of Recorder 6 where the SQL code has never been re-optimised for SQL Server (we use a translator to make sure the SQL at least runs) but we are gradually updating to optimised SQL as work is done on each part of the system.

John van Breda
Biodiverse IT

7

Re: SQL server questions

I am slightly unsure of the issues surrounding SQL server + MSDE for a network installation. We use Netware servers at CCW, the documentation mentions that MSDE would probably sit on a file server for a network install but later mentions a minimum spec for a MSDE server of windows 2k or above. Is this referring to a file server or a database server or either?

Stupid question 1) will MSDE "work" on a netware file server or does it have to be windows box?
Stupid question 2) Does win2k refer to win2k desktop or win2k server?

CyprianPayne
CCW

8

Re: SQL server questions

MSDE only runs on Windows, as far as I am aware. When it refers to Win2K, it means either the server or the desktop editions - it doesn't matter which.

Just to clarify matters, MSDE (Microsoft SQL Server Desktop Engine) is, essentially, a cut-down version of the full SQL Server 2000 software. Also, the difference between a database server and a file server is really all about how they're configured and what software they have installed on them, rather than them being intrinsically different beasts. In other words, a file server that you install MSDE or SQL Server onto so becomes a database server as well as a file server.

Hope that helps!

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

9

Re: SQL server questions

Something to clarify - putting the database server onto the file server will work fine, but for optimal performance having a dedicated file server and a separate dedicated SQL Server box is desirable.  For most users of Recorder 6 this probably make a lot of difference though.

John van Breda
Biodiverse IT

10

Re: SQL server questions

We currently run our SQL Server 2005 install on our mirror backup server (It's a 3GHz single Xeon Dell PowerEdge). The server pretty much idles there the whole time doing nothing, just copying files from the main server occassionally, so we thought we'd make better use of its resources and get Recorder on there. It been a very satisfactory arrangement so far, especially since we upgraded it to 3GB of RAM. Even with Recorder on there, it still sits there idling most of the time as even with the potential for 7 concurrent users, only one person ever seems to be using it at one time.

A couple of points woth noting from all of this:

1) one of the biggest advantage we have found in putting our Recorder db onto a seperate server is ease of maintenance. The server is more simple overall, so less complexity means there is less to go wrong and generally makes it much easier to maintain.

2) Even though the server idles for roughly 95% of the time (at a guess), those big Recorder reports can really tie it up for the other 5% of the time. When we had Recorder on our general purpose server, other people within the organisation would notice and start complaining when we were running large reports. Now that Recorder is on a semi-dedicated server, we only have to worry about slowing down other Recorder users.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

11

Re: SQL server questions

Thanks for that last post Charles -very useful info for our networks guys.

Do you know what the implications of using 5 or 6 different nbndata files on a single networked sql server would be? (or if its even possible or the way to go about things). We now have many different standalone R2k2k databases: bats, bugs, herps, etc. should we dump them all into a single "bucket" networked SQL nbndata file or have different backends (nbndata files) for each one -but all on the same SQL server?

CyprianPayne
CCW

12

Re: SQL server questions

Cyprian,

What volume of data are you storing in each of the databases?

John van Breda
Biodiverse IT

13

Re: SQL server questions

we will be converting our inverts db from R3>R1K2>R6 -it is the largest db we have

sizes:

bugs database ~ ?MB (in R3)
10k locations
3.5k people
4k references
500k records

grasslands database: R2k2 ~ 178 MB nbndata file
240k species
20k habitats
1k locations

there are at least 4 more habitats databases to convert - guesstimate about 100 Mb each

CyprianPayne
CCW

14

Re: SQL server questions

It sounds like you will have a million or a bit more records.  Recorder 6 should have no significant problems coping with this amount of data in a single dataset.

John van Breda
Biodiverse IT

15

Re: SQL server questions

What about reporting by dataset and exporting  to the NBN? wouldnt it be easier to keep the databases on the same SQL sever but seperate from each other?

CyprianPayne
CCW

16

Re: SQL server questions

This is one of those features that I'd like to see go on the wishlist - the categorisation of surveys within Recorder so that sets of surveys can be grouped together in some sort of hierarchical folder-like structure. This would then make it not only very easy to keep distinct sets of data seperate from each other, but would also allow us to combine them just as easily. The ability to 'fence-off' certain sets of data is become increasingly important for us. For instance, there is an increase in the number of cross-county-border enquiries we need to do. This requires obtaining records from other centres, so we'd like to be able put other counties' data into their own folders.

Regarding the reporting side of things, now that we have the taxon group information in Recorder, the ability to filter on a particular group would come in very handy, although it's easy enough to do using XML Reports. Are you able, or do you have anyone able, to write SQL queries for you Cyprian? Having these skills in-house, or available on tap really opens up endless possibilities.

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

17

Re: SQL server questions

charlesr wrote:

This is one of those features that I'd like to see go on the wishlist - the categorisation of surveys within Recorder so that sets of surveys can be grouped together in some sort of hierarchical folder-like structure.

This is also something I would like to see, to allow us to group surveys, possibly imported from different organisations, separately from our own dataset, keep the whole "filing system" less cluttered and make items in the survey/event hierarchy easier to find. I would prefer this top level of survey groups not to be locked to the ID number, for maximum flexibility.

Gordon

Gordon Barker
Biological Survey Data Manager
National Trust

18

Re: SQL server questions

Gordon or Charles, would either of you like to submit the above in the "feature request" thread?

CyprianPayne
CCW

19

Re: SQL server questions

Hi All,

Can i just clarify please exactly what is the optimum set up for Recorder 6? 

This year we purchased a dedicated server for use by Recorder 6.  The 1st setup involved both the program files and the SQL database both being placed on this new server.  Import and export times were reduced and all was well :).  Recently in the 2nd setup i've moved the program files to a different server (other programs are installed here too) so Recorder 6 is running as this extract from the Recorder 6 network set up guide suggests:

It is most common for SQL Server to be installed on a dedicated server and the installation of Recorder 6 allows and supports this. It is possible to store the file server files on the same machine as the database server (SQL Server), although for best performance this is not recommended.

Therefore i've moved the file server files to a separate server and the new server has become a dedicated database server. 

The problem is I have now noticed an increase in import, export and reporting times when I had expected to see even faster completion of such tasks.  Have I got the wrong end of the stick here, or have I missed something?

Thanks,
Catherine

Catherine

20

Re: SQL server questions

from what I understood -I could be wrong- if you only have R6 &  SQL on the server then it is best to have everything on the one box, but if the server is also acting as a non R6 file server for use then it is best to seperate SQL.

CyprianPayne
CCW

21

Re: SQL server questions

We originally had our R6 setup on our main fileserver which is very busy and rather overcrowded, particularly in terms of memory. So we moved it (both SQL Server and the Recorder program files) to a seperate server, which isn't as powerful in terms of CPU as the main file server, although it does have more RAM. This second server does various other things, like mirroring the files on the main file server, so it's not completely dedicated to R6. But there was a significant performance enhancement, mainly due to the increased RAM (we've got 3GB in there). It's all a matter of scale, though. If you've got a relatively underused file server, with only a few people using it, then you're not going to see much of a performance hit by housing everything togther. I suspect that advice in the network setup is aimed at very large organisations, with tens- if not hundreds of users (like Thames Water perhaps).

With importing, times are very much affected by the client, i.e. the computer you're working on. A lot of the processing is done locally, rather than on the server, so if you've got a slow client, a faster server won't help.  I don't see why reporting would be affected at all by moving the program files, or exporting. What speed is your network? Is there a lot of traffic? I'd actually recommend keeping the whole Recorder setup together on the same machine to reduce network traffic, unless there's a provable reason this setup slows Recorer down.

Charles

Charles Roper
Digital Development Manager | Field Studies Council
http://www.field-studies-council.org | https://twitter.com/charlesroper | https://twitter.com/fsc_digital

22

Re: SQL server questions

Hi and thanks for the prompt replies. 

On Friday I moved the Rec6 install again so now the SQL and the program files are back on a completely dedicated server again. 

After some testing I notice processing speed has increased again.  I'm thinking the server i'd placed the program files on is just too over used and to have everything on this one dedicated server is by far the better option for now. 

Thankfully i've got Rec6 installation here down to 1hour so it hasnt been too much hassle. 

Thanks again for the info and advice,
Catherine

Catherine

23

Re: SQL server questions

Am currently writing our system manual & would be interested in (any) successful complete network install details:

How much did you need to deviate from the "installation guide"?
Would you be able to send me a few pointers which are missing from it , but you had to do?

Many thanks in advance,

Cheers now, Rob.

24

Re: SQL server questions

I too would appreciate any information users can provide in this area - to help with the improvement of the network installation guide.

Would it be possible for you to copy me in on anything you can send?

Many many thanks,

Sarah

Sarah Shaw
Biodiversity Information Assistant
JNCC

Sarah Shaw
Biodiversity Information Assistant
JNCC

25

Re: SQL server questions

We are thinking of upgrading to Recorder 6, as is one of Devon Wildlife Trust's Reserves Officers (Steve).  We currrently have Recorder 2002 as a network installation, he has his own standalone version.  We will have a new server with SQL server installed.  Will Steve be able to run his version of R6 with remote access to our server, or should he get his own SQL server licence?  (He is based in the wilds of North Devon, a long way from Exeter).  Does R6 come as network & standalone systems?

I hope these questions make sense - our IT Manager asked me to find out, and I don't alway understand what he is talking about!

Ellie

Eleanor Bremner
Senior Records Centre Officer
Devon Biodiversity Records Centre