Web3.0 Commerce Ontology, Knowledge Graph and NLP Search Engine Experiment

-------------------------------------------  Draft:   starting 6/16/2023 and ending 7/20/2023  ------------------------------------------

This experiment focuses on eCommerce NFT Asset and Offering discoverability.  

Our goal is to develop a working prototype of NFT based eCommerce catalog ontology, catalog knowledge graph, blockchain contracts, and natural language search service (figure below).

 Key Hypotheses:

• Blockchain NFT Assets (Utility NFT) will eventually have traits (metadata) that follow a common semantic ontology

• Blockchain NFT Assets will be organized into catalog categories. This hierarchical categorization will be managed on-chain.

• Asset Offerings will be managed on-chain as NFT Offerings. These Offerings will follow a common ontology

• Blockchain NFT Asset and Offering metadata will be available off-chain in a Decentralized Knowledge Base

• AI Natural Language Question and Answer engine will leverage Catalog Decentralized Knowledge Base data. 

• Third party apps want to discover and access NFT Assets and Offerings directly without having to deal with a centralized authority like Opensea where this data is stored off-chain.

• Future commerce eco-systems will be focused on particular domains. These catalog focused commerce solutions will differentiate from larger cross domain solutions like amazon.

Capabilities

• NFT asset and offering discovery using natural language search engine

• Catalog hierarchical categories, assets, and offerings direct query via SPARQL endpoint

• Direct blockchain access to on-chain hierarchical category, asset, and offering NFT's

This experiment incorporates a Natural Language search engine that leverages this NFT ontology, knowledge base, and catalog taxonomy.

We hope that, through this experiment, we can get involved with the POC4Commerce Hackathon. Most parts of this experiment require a robust NFT ontology.

Experiment Design

The Web3.0 and semantic metadata industry has built robust solutions that have many layers of abstraction to support all types of supply chain use-cases.  Many of these solutions see eCommerce as a Supply Chain Variant and expect developers to understand how to leverage these abstract frameworks.

NFT industry has gained popularity because of the simplicity of their solutions and metadata models championed by OpenSea.

We are going make the design and implementation as concrete and simple as possible. This solutions will focus entirely on the eCommerce use-case.

We started our journey by building abstract smart contracts and ontology that could be used for a wider set of supply chain use cases and found that things got complicated quickly.   As we show the software design we will refer back to industry patterns and practices that are used in the larger supply chain industry.

Solutions is not built around a single coin or based on a central service API. 

Approach

• Make ontology, blockchain contracts, off-chain metadata, RDF structure, and search engine as simple as possible

• Use concrete domain vocabulary and not multiple levels of language abstraction

• Provide usable smart contracts to 3rd party developers

• Follow industry best practices where possible

Blockchain Contracts

Catalog , category hierarchy, assets and offerings will all be on-chain NFT's. Each NFT Entity (Catalog, Category, Asset, and Offering) will hold off-chain JSON-LD semantic metadata. 

Catalog Contracts all derive from base ERC721 standard extension Contracts to support a Decentralized Category Knowledge Graph (CKG) taxonomy. Our experiment Catalog leverages a more abstract CKG contract framework.

Having these Entities on-chain will allow for 3rd party applications to have full access to the catalog taxonomy and information.

This approach can integrate with opensea seaport approach for offer settlement. Seaport's centralized approach of holding all offering entities off-chain prevent third parties from direct access and discovery of offerings. Opensea also holds catalog and category hierarchy taxonomy off-chain which makes it difficult for collaborative creation of the catalog.

Ontology

NFT off-chain JSON-LD metadata and Decentralized Knowledge Base will follow a standard Ontology. This ontology will follow industry best practices and be derived from schema.org, goodrelations, POC4Commerce schema.

Our ontology creation process follows the recommendations by Noy and McGuiness. 

The modelling and creation of the ontology followed an iterative process and was done using Protégé. The applied steps can be summarised as follows: 

1. In the first step we determine the domain and scope of the ontology. 

2. We gather literature about distributed ledger technologies, their components and existing standards, application domains, security vulnerabilities and threats (cf. Sec. 3.2). 

3. We define a set of questions “that a knowledge base based on the ontology should be able to answer.” [5] We derive use cases and create competency questions based on the collected information (cf. Section 3.3). 

4. We create an ontology to express the collected information (cf. Section 4). 

5. We evaluate the ontology by providing SPARQL queries to answer the collected competency questions. 

6. Finally, we instantiate the developed ontology with named entities and relations extracted from the literature (cf. Section 4.1). 

Decentralized Knowledge Base

Our Knowledge Base is a hybrid Ethereum and IPFS solutions. All knowledge base graph nodes are represented as Ethereum NFT Entities where each Entity maintains a collection of children Entities (connections). All NFT Entity metadata is held off-chain in IPFS JSON-LD.

This approach allows 3rd party applications direct access to the full knowledge graph of data. This solution maintains an off-line read-only copy of the catalog graph to RDF online storage.

Natural Language Question and Answer Service

Natural Language QA service will embed Knowledge Base information within a ChatGPT LLM framework using the LangChain platform. We are leveraging the GPT4ALL framework.

Tasks

• Build Contract Factories for Taxonomy Catalog, Category, Asset, and Offer.  Build out Entity Contract that supports collections and base behaviors

• Write Decentraland Items (Entities) onto blockchain

  • write to off-chain metadata using either OpenSea Traits standard, JSON-LD standard

  • not write out in RDF standard like EIP-6239 recommends.  JSON-LD much easier to work with.

• Build Catalog ontology (OWL files)

• Construct RDF store (RDF files at first) from blockchain data.  Traverse from top Catalog NFT

• Build set of SPARQL queries of RDF data in local datastore

Write Decentraland items onto blockchain

Off-chain metadata approach

JSON-LD metadata format. link   Basically extend current OpenSea attributes to include JSON-LD information

Address interoperability issues of OpenSea metadata representation.  Need way of presenting intended meaning of data.

{

  "@context": "https://json-ld.org/contexts/person.jsonld",

  "@id": "http://dbpedia.org/resource/John_Lennon",

  "name": "John Lennon",

  "born": "1940-10-09",

  "spouse": "http://dbpedia.org/resource/Cynthia_Lennon"

}

Resource Description Framework (RDF).  

Token metadata reference to RDF storage.

Example, Decentralized Knowledge Graph (DKG) from orgintrail.io.  working with ontochain and tracelabs.  Leverages a layered architecture link.  Everything accessible via a centralized service layer.  TRAC ERC20 token

Issues:

• single centralized service to DKG search (instead of many catalog NFT on the blockchain)

• single centralized RDF storage mechanism

Example, for social network representation. Semantic Soulbound tokens (EIP-6239).  Extends ERC-721 and ERC-5192.

Jessica Chang j.chang@relationlabs.ai 

Companies using this approach

relationlabs.ai

Building Contract Factory and Contracts

Leverage Singleton, Collection, and Item Contract Factory Solution.  There will probably be something like this in the future to support collection hierarchies.

Nested Contracts (Parent and Child Tokens)

• Cross Contracted Nested (ERC-6059)

• ERC-6150 Hierarchical NFTs (within single contract), ERC-6151 extend this to token bound NFT's

• ERC-5606: Multiverse NFTs.  Bundle and unbundle NFT's

• Wrap NFT Token with other NFT that manages hierarchy

• EIP-3652: Hierarchical NFT - Tokens (???)

• Hierarchy via composability ERC6220.  ERC-6220: Composable NFTs utilizing Equippable Parts.  Used for catalog use-cases. ERC998

• “Soul-Binding” of assets within the NFT wallet.  ERC-5114: Soulbound Badge. Cannot be transferred once created or acquired.

good article on nested NFT's link

Building Catalog Developing Knowledge Graph

There are several tools that help you construct Asset NFT JSON-LD from your current product catalog content.

Relation labeling in E-commerce with LLMs 

LLM's to auto-create relationships between products, variants, assets. 

article

• complementary or substitutable relations between products, variants, assets

Using LLM to drive shopping is an essential element of an eCommerce Digital Experience.

Establishing Customer Shopping Behavior using LLM

Amazon article, FolksScope initiative

NFT Marketplaces

publish or seek for a certain asset with specific supply chains and at certain conditions

Terms

Offering (Order)

An or­der is an ac­count-sub­mit­ted in­tent to buy/sell an as­set. It ex­ists off-chain, and has a type: bid or ask. It also con­tains meta­da­ta that is re­quired to match to NFTs on-chain. There are two par­tic­i­pants in an or­der: a mak­er, who makes liq­uidi­ty and acts as a first mover in a sale by declaring intent to sell or buy/bid on an item, and a tak­er, who takes liq­uid­i­ty and acts as the sec­ond mover in a sale by buying or accepting a bid on the item. Orders can be bundled within a specific collection. 

Offerer (both buyer and seller)

Listings (seller creates orders)

list of orders

Offers (buyer creates orders)

list of orders

Fulfiller

Fulfilling an offer.   Seller providing NFT and stuff to buyer.  Fulfiller account must own the NFT

Fulfilling a listing. Buyer pays for NFT.  The response contains all of the information needed for clients to fulfill a listing directly on chain (pay for NFT) 

Maker/Taker  (either seller or buyer can initiate an order)

Maker initiates the order

Taker fulfills the order 

complete any number of listings at once via a set of fulfillments 

A zone is a contract that performs additional validation before the fulfillment 

A conduit is a type of contract where offerers specify token approvals.

Order types

• Full orders — those that do not support partial fills

• Partial orders — enabling the filling some fraction of the order (each item must be divisible by the supplied fraction)

• Open orders — the call to execute the order can be submitted by any account

• Restricted orders — those that should be executed by the offerer or the zone of the order

OpenSea seaport protocol (v1.5) 

Seaport is a marketplace contract for safely and efficiently creating and fulfilling orders for ERC721 and ERC1155 items. Each order contains an arbitrary number of items that the offerer is willing to give (the "offer") along with an arbitrary number of items that must be received along with their respective receivers (the "consideration"). 

Single Seaport contracts

To create an order

1. You sign an off-chain listing as the offerer

2. Validate the on-chain order as the offerer

3. Contract that can serve as an offer (dynamic order)

Fulfilling order (fulfill orders)

Give the whole payload and creates a mirror order (fulfill orders)

Buy an NFT,  Sell an NFT and pay a fee (other type of fulfilling usually used for fee's).  (Fulfill available orders) method

• Fee:  Non-NFT consideration

(Match orders).  not utilized that often but used in powerful use-cases. (power feature)

Zones.  check the order prior to fulfillment.  For example Dynamic NFT Metadata.  Owner changes the metadata while your waiting for fulfillment.   

Listing has a ERC712 signature payload

Marketplace capabilities

• Dutch auctions — a selling strategy where a price is lowered until a customer buys an asset. 

• Bargaining

• Auctions

• Valuations

• Price Determination Mechanisms

• Bartering

• Bundling

• Tipping

• ERC2981.sol royalty contract

OpenSea Creatures

OpenSea use of Contract Factory to mint NFT's

https://github.com/ProjectOpenSea/opensea-creatures/blob/master/contracts/ERC721Tradable.sol

Diagram of order flow

https://github.com/ProjectOpenSea/seaport/tree/main/diagrams

Soul bound stuff

Soulbound Badges (EIP-5114) takes a similar strategy of building out a very concrete solution for a particular use-case.

Application Configuration

python.exe -m pip install --upgrade pip

pip install python-dotenv

Blockchain Ontology - Semantic Net (Knowledge Graph)

Some key features of Web 3.0 include:

• Use of semantic technologies: Web 3.0 uses technologies such as RDF (Resource Description Framework) and OWL (Web Ontology Language) to give meaning to the data on the web, making it easier for machines to understand and process.

• Linked data: Web 3.0 aims to create a web of interconnected data, where different sources of information are linked together and can be easily accessed and analyzed by machines.

• Personalization: Web3.0 technologies allow for more personalized experiences on the web, as they can gather and process data about an individual’s interests and preferences.

• Improved search capabilities: Web3.0 technologies allow for more accurate and sophisticated search capabilities, as they can understand the meaning of the data being searched for and provide more relevant results.

https://techlabuzz.com/web-3-0-the-next-generation-of-the-world-wide-web/

The goal of the Semantic Web and Linked Data is to create a more intelligent and connected web, where data can be easily shared, integrated, and analyzed. 

• Linked Data

• Interoperability

https://www.unitcelleducation.com/2023/02/What-is-web-3-future-of-the-Internet.html

Value

• More trustworthy because of decentralized public records  

• No more dependent on centralized authorities and data repositories  

• Personalized interactions with users  

• Faster and superior search results driven by AI  

• No more dependency on mediators  

• More peer-to-peer communication and connectivity 

Blockchain and the Semantic Web

https://www.delta-h2020.eu/wp-content/uploads/2020/01/Towards-blockchain-and-semantic-web.pdf

Juan Cano-Benito(B) , Andrea Cimmino(B) , and Ra´ul Garc´ıa-Castro(B) Universidad Polit´ecnica de Madrid, Madrid, Spain 

• simple knowledge organisation system (SKOS) 

• web ontology language (OWL) 

• resource distribution framework (RDF) 

• RDF Schema (RDFS)

Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6 - 95125 - Catania, Italy

unict.it

Giampaolo Bella, a Domenico Cantone, a Carmelo Fabio Longo, a Marianna Nicolosi Asmundo a and Daniele Francesco Santamaria a,∗ a 

OASIS v2, June 2023

• OC-Found (that includes OASIS) for agents (commercial participants, smart contracts). 

• OC-Commerce for commercial offerings, assets and activities. 

• OC-Ethereum for Ethereum smart contracts and related tokens (compliant with ERC20, ERC721, and ERC1155). 

Blondie 1.0 (7 years started, updated 4 years ago)

• https://github.com/hedugaro/Blondie/tree/master

• https://hedugaro.github.io/Linked-Blockchain-Data/

• https://semanticblocks.wordpress.com/

OASIS V1 Ontology

OASIS V2 Ontology

overview https://arxiv.org/pdf/2306.10061.pdf   June 14, 2023

SPARQL Query of NFT Instance

PREFIX rcether: <http://localhost:5000/owl/rc-ethereum.owl#>

PREFIX rccom: <http://localhost:5000/owl/rc-commerce.owl#>

PREFIX oasis: <http://localhost:5000/owl/oasis.owl#>

PREFIX gr: <http://purl.org/goodrelations/v1#>

PREFIX rcgen-test: <http://localhost:5000/owl/rcgen-test.owl#>

SELECT DISTINCT ?nft ?tokenId ?offering ?priceDetermination ?value ?currency WHERE { 

    ?nft a rcgen-test:EthereumTokenERC721.  

    ?nft rcether:hasTokenID ?tokenId.

    ?offering rccom:isOfferingAbout ?nft.

    ?priceDetermination a rccom:PriceDeterminationActivity.

    ?priceDetermination rccom:priceDeterminationPerformedOn ?offering.

  ?priceDetermination rccom:hasPriceValue ?priceValue.

    ?priceValue gr:hasCurrencyValue ?value.

    ?priceValue gr:hasCurrency ?currency.

}

NFT Ontology

https://rhizomik.net/reports/semantic-nfts

Metaverse Ontology

https://www.frontiersin.org/articles/10.3389/fdata.2022.998648/full

Companies working in this area

Trace Labs

https://tracelabs.io/join/decentralized-knowledge-graph-engineer-Mjg=

 working with ontochain link

46-48 Wyndham St, Central Ljubljana, Slovenia 

$9M annual revenue

Decentralized Knowledge Graph

https://origintrail.io/