Hi Matt
That’s true, there is an additional data access control which enables data providers to put datasets on the Gateway with no public access at all.  This control was created for use in the following circumstances:
1.    Datasets that are still undergoing verification may be loaded on the Gateway with no public access as a temporary measure until the verification process is complete and public access can be granted.  One of the Scottish LRCs is currently using this control for this purpose.
2.    Datasets which consist entirely of records which are too sensitive to allow public access even at 10km resolution.  For example, some RSPB single species breeding records datasets. 
3.    Demonstration or test datasets, such as the one we use to demonstrate Record Cleaner at training sessions.
The advantage of putting these datasets on the Gateway despite the lack of public access is that access can be granted to key data users.
We do not plan to remove this control, i.e. it will still be possible to share datasets via the Gateway with no public access under the proposed new system of access controls.  However, we do plan to monitor its use more closely to ensure it is only being used in the circumstances described above.  Most importantly, we will encourage data providers who use this control to proactively grant access (i.e. at the data loading stage) to a wider group of key data users, which would include all statutory agencies and local environmental records centres.
There are currently 1.6 million records from 22 data providers with no public access via the Gateway.
You’re right, I don’t think this control has been discussed during the consultation so far, so thank you very much for raising it! 
Best wishes
Paula

Regarding the access to datasets, I'm quite happy that there are controls on the access levels to individual datasets, some of the datasets I manage have limited access (eg public access is limited to seeing records at a 1km level so as to "blur" the data for sensitive species such as Great Crested Newt). 

The point I am making is that if you look at NBN maps on a regular basis you notice that there are a number of LRCs that have submitted data where apparently the entire submission it tagged as "dataset unavailable", even for common species, if I or anyone else wanted to look at a distribution at 10k level we would have to apply for access to those datasets.  Look at a map of Episyrphus balteatus of you want an example of the sort of thing I am talking about.  Surely ALL data submitted to the NBN should be visible at the 10k level by default except for the records of the few sensitive species as noted above.

I think Charlie's second suggestion may be more useful viz. the ability to hide datasets existence from the eyes of the general user of the Gateway with the data provider proactively giving users/organisations access.

Since the stat agencies are requiring the use of the Gateway as a delivery mechanism, some LRCs are getting round the problem of refusals from individual recorders to put data on the Gateway (in a publically available sense) by creating datasets of these records and only allowing access to their partners. This is usually not the LRC themselves being difficult btw! As well as this case, there are other reasons for records being placed in "hidden" datasets, such as them being known duplicates of those uploaded by a national scheme (but perhaps not quickly enough or detailed enough for the stat agencies) or unverified as previously mentioned.

I don't think an assumption that records wouldn't be useful to others is ever a reason for non-public access datasets, and reasons of species sensitivity/environmental harm are probably in the minority.

It could be argued that the NBNG is not the best mechanism for this kind of non-public disemmination, but we are where we are and organisations have bought into the Gateway and developed tools to use the data, so the number of these non-public datasets is likely to increase rather than decrease in the short-term at least.

To debate with myself, I can see a reason for not hiding these datasets - the signposting opportunities that the Gateway offers for the existence of pertinent records to researchers and specialists. I think all the datasets mentioned here do clearly explain the situation in their metadata pages so perhaps the status quo is fine?

I think we are one of the "best" LRCs for putting data on the Gateway in the sense of putting a high proportion of our records on, but probably one of the "worst" compared to LRCs who only put their really super high quality, comprehensively verified data on. Regardless, I am concerned that records that fall into the categories above which we make available to partners offline will not be available to our national partners so will probably be uploading a "hidden" dataset soon of those records.