Adept Search

/ User Data Functions

ABOUT

This Automated Data Extraction PlaTform is a set of webtools to facilitate access to the xDD library of full-text documents (https://xdd.wisc.edu). xDD is the largest collection of open and controlled-access scientific articles available for bulk text data-mining in the world.

Machine-learning researchers can use ADEPT to:

1. SEARCH xDD metadata records
2. Create a SAVED SET of one or more search results
3. Request a TEST SET of 200 random documents from a saved set for local development purposes.
4. Submit APPLICATIONS for high-throughput deployment on the entire xDD library
5. Share saved sets, test-sets, and applications with GROUPS of collaborators

SAVED SETS

A user may save the results of one or more metadata searches as a SAVED SET. Saved Sets are created in the saved sets tab, then populated by saving searches and citations on the Search page. The search parameters in a saved set can be also be used to access the xDD API directly to access the same data directly from the source.

TEST SETS

xDD must control access to its resources to ensure that they are used for legitimate scientific research in bulk text data-mining and that xDD is not used by bad-actors to circumnavigate normal licensing restrictions (i.e., to download PDFs that they would not normally have legal access to for purposes other than text data-mining). xDD fulfils this responsibility by ensuring that its full-text documents do not leave its servers.

An exception to this rule are test-sets that ADEPT users may use for local development of xDD APPLICATIONS. A test-set is a random sample of 200 documents from a SAVED SET that a user may download onto their local machines for development purposes. New test sets can be requested from the SAVED SETS or DICTIONARIES tab. Once approved by the xDD system, users can find a unique API-key and URL to to their requested test-set in the TEST SETS tab.

DICTIONARIES

Dictionaries are a collection of search terms, and can be either xDD identified Dictionaries, or user constructed dictionaries. Dictionaries can also be used as the source for the creation of TEST SETS, which are a data product identified in the GeoDeepDive system used in ML processing.

APPLICATIONS

ADEPT defines an xDD application as a dockerized set of machine-learning scripts for the purpose of bulk text data-mining. xDD administrators need to monitor whether a submitted text data-mining application is respectful of its Terms of Service. Users can use the applications tab to submit an application for review and approval by an xDD administrator or to request that an already approved application be deployed on the xDD high-throughput computing architecture.

TERMS OF SERVICE

Background
These Terms of Service (hereafter TOS) define the responsibilities of Automated Data Extraction Platform (hereafter ADEPT) account holders (hereafter USERS) with respect to proper and acceptable use of its resources. ADEPT is the front-end interface for accessing the xDD corpus of digital documents. xDD was originally supported by the U.S. National Science Foundation EarthCube building block project (NSF ICER 1343760). Additional ogoing critical support is being provided by DARPA ASKE HR00111990013. Additional support is provided by the DOE and USGS. A through-going objective of all of these projects is to build a cyberinfrastructure that is capable of supporting end-to-end text and data mining (TDM) and knowledge base creation/augmentation activities in any domain of science or scholarship. xDD infrastructure includes the following key components:
1. Automated, rate-controlled and authenticated original document fetching
2. Secure original document storage and bibliographic/source metadata management
3. Automated pre-processing of documents by multiple software tools; ability to quickly deploy new tools/versions of tools across all documents
4. API for basic full-text search and discovery capabilities
5. Ability to pre-index content using external dictionaries
6. Ability to generate fully documented, bibliographically complete testing and development datasets based on user-supplied terms
7. Capacity to support the deployment of user-developed TDM applications across full corpus, with on-demand updates as new relevant documents are acquired

Both ADEPT and xDD are managed by the University of Wisconsin-Madison.

Valid Users

ADEPT is meant to support the design and development of machine-learning software applications for automated knowledge base construction by non-profit researchers. USE OF THESE RESOURCES BY FOR-PROFIT INSTITUTIONS IS NOT PERMITTED WITHOUT OBTAINING ADDITIONAL PERMISSIONS BEYOND THE SCOPE OF THESE TOS AND INQUIRIES FOR COMMERRCIAL USE MUST BE DIRECTED TO contact@geodeepdive.org.

Compliance
USERS are responsible for complying with this TOS and for violations thereof attributable to them. ADEPT reserves the right to monitor and investigate usage and content at its sole discretion to identify possible violations of these TOS, and/or to protect ADEPT resources and its user community. ADEPT EXPRESSLY DISCLAIMS ANY LIABILITY FOR THE CONTENT AND DATA TRANSMITTED THROUGH OR INTERMEDIATELY, TEMPORARILY, OR PERMANENTLY STORED ON ANY SERVER OR VIRTUAL MACHINE AND FOR THE ACTIONS OR OMISSIONS OF USERS.

User Security Obligation
USERS must use reasonable care to ensure the security of their account, uploads, images, virtual machine (VM), etc. A User is solely responsible for any intrusions into or security breaches of their accounts, except as otherwise covered by a specifically designated security administration or firewall security service package. ADEPT reserves the right to discontinue any submitted software applications, without notice, on its resources as a result of a security compromise.

Prohibited Content
USERS shall not post, transmit, or store data or content on or through software applications submitted to ADEPT servers which, in ADEPT's sole determination, constitutes a violation of any federal, state, local, or international law, regulation, ordinance, court order or other legal process. Prohibited content includes, without limitation, content or code that:
+ Facilitates any violation of these TOS
+ Includes 'harvested' addresses or information
+ Includes phishing websites or spamvertising sites
+ Includes material believed by ADEPT to constitute pornography, pirated software, hackers' programs or archives, Warez sites, IRC bots, MP3 files, spyware of any sort, Spam software of any sort, etc.
+ All ADEPT hosting services, virtual or otherwise, may only be used for lawful purposes.

Abuse and Unacceptable Uses
USERS are prohibited from engaging in any activities that ADEPT determines, in its sole and absolute discretion, to constitute abuse or unacceptable uses, including but not limited to:
+ USERS may not submit any software applications to the ADEPT infrastructure that transmit, distribute, download, copy, cache, host, or store any information, data, material, or work that infringes the intellectual property rights of others or violates any trade secret right of any other person/User.
+ Introducing or executing malicious content or programs into ADEPT servers or virtual machines, such as viruses, worms, Trojan horses, and key loggers
+ Causing or initiating security breaches or disruptions of system communications and/or connectivity, including port scans, flood pings, email bombing, packet spoofing, IP spoofing, and forged routing information
+ Executing any form of system activity that will intercept data not intended for the USERS's account
+ Evading or circumventing USER authentication or security of any host, network, or account, including cracking, brute-force, or dictionary attacks
+ Interfering with or denying service to any USER, host, or network other than the User's host, such as a denial of service attack or distributed denial of service attack
+ Conduct designed to avoid restrictions or limits access to specific services, hosts, or networks, including the forging of packet headers or other identification information
+ Soliciting the performance of any illegal activity, even if the activity is not performed.
+ Using any program, or sending messages of any kind, designed to interfere with or disable a User's terminal session
+ Revealing your account password to others or allowing use of your account by others
+ Using ADEPT's computing resources to actively engage in procuring or transmitting material that is in violation of sexual harassment or hostile workplace laws in the User's local jurisdiction
+ Engaging in activities, whether lawful or unlawful, that ADEPT determines to be harmful to its Users, User community, operations, reputation, community goodwill, or community relations
+ Running applications or services to mine cryptocurrency and/or that use block-chain technology for personal gain. Exceptions must be requested in advance and will be approved at ADEPT's discretion.

Violations of TOS
ADEPT reserves the right to deactivate a User's account(s) if any or all portions of this TOS are being violated in any way, in ADEPT's sole and absolute discretion.

Indemnification
USER agrees to indemnify and hold harmless ADEPT and its staff, directors, agents, and collaborators from and against any claims, losses, judgments, damages, costs, and expenses incurred by any of the foregoing parties due to or resulting from:
+ USER's publication of unacceptable content or materials stored on xDD resources
+ USER's download of prohibitied content or materials stored on xDD resources
+ USER's breach or violation of any covenant contained in this policy, or resulting from their use of ADEPT
Close