rosicast

Notation

  • Notation
    • Algebra
    • Calculus
    • Linear algebra
    • Probability
    • Set theory
    • Statistics

CS224w_2021_fall

  • CS224W - Colab 0
    • NetworkX Tutorial
      • Setup
      • Graph
      • Node
      • Edge
      • Visualization
      • Node Degree and Neighbor
      • Other Functionalities
      • Documentation
    • PyTorch Geometric Tutorial
      • Installing dependencies
      • Visualization
      • Introduction
      • Dataset
      • Data
      • Edge Index
      • Implementing Graph Neural Networks (GNNs)
        • What is the output of a GNN?
        • Training on the Karate Club Network
      • Documentation
  • CS224W - Colab 1
    • 1) Graph Basics
      • Setup
      • Zachary’s karate club network
      • Question 1: What is the average degree of the karate club network? (5 Points)
      • Question 2: What is the average clustering coefficient of the karate club network? (5 Points)
      • Question 3: What is the PageRank value for node 0 (node with id 0) after one PageRank iteration? (5 Points)
      • Question 4: What is the (raw) closeness centrality for the karate club network node 5? (5 Points)
    • 2) Graph to Tensor
      • Setup
      • PyTorch tensor basics
      • Question 5: Get the edge list of the karate club network and transform it into torch.LongTensor. What is the torch.sum value of pos_edge_index tensor? (10 Points)
      • Question 6: Please implement following function that samples negative edges. Then answer which edges (edge_1 to edge_5) can be potential negative edges in the karate club network? (10 Points)
    • 3) Node Emebedding Learning
      • Setup
      • Visualize the initial node embeddings
      • Question 7: Training the embedding! What is the best performance you can get? Please report both the best loss and accuracy on Gradescope. (20 Points)
      • Visualize the final node embeddings
    • Submission
  • CS224W - Colab 2
    • Device
    • Setup
    • 1) PyTorch Geometric (Datasets and Data)
      • PyG Datasets
      • Question 1: What is the number of classes and number of features in the ENZYMES dataset? (5 points)
      • PyG Data
      • Question 2: What is the label of the graph with index 100 in the ENZYMES dataset? (5 points)
      • Question 3: How many edges does the graph with index 200 have? (5 points)
    • 2) Open Graph Benchmark (OGB)
      • Dataset and Data
      • Question 4: How many features are in the ogbn-arxiv graph? (5 points)
    • 3) GNN: Node Property Prediction
      • Setup
      • Load and Preprocess the Dataset
      • GCN Model
      • Question 5: What are your best_model validation and test accuracies?(20 points)
    • 4) GNN: Graph Property Prediction
      • Load and preprocess the dataset
      • Graph Prediction Model
        • Graph Mini-Batching
        • Implemention
      • Question 6: What are your best_model validation and test ROC-AUC scores? (20 points)
      • Question 7 (Optional): Experiment with the two other global pooling layers in Pytorch Geometric.
    • Submission
  • CS224W - Colab 3
    • Device
    • Installation
    • 1) GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GraphSage Implementation
      • Building Optimizers
      • Training and Testing
      • Let’s Start the Training!
      • Question 1.1: What is the maximum accuracy obtained on the test set for GraphSage? (10 points)
  • CS224W - Colab 4
    • Device
    • Installation
    • 1) GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GAT Implementation
        • Multi-Head Attention
      • Building Optimizers
      • Training and Testing
      • Let’s Start the Training!
      • Question 1: What is the maximum accuracy obtained on test set for GAT? (10 points)
  • CS224W - Colab 5
    • Device
    • Installation
    • DeepSNAP Basics
    • 1) DeepSNAP Heterogeneous Graph
      • Question 1.1: Assigning Node Type and Node Features
      • Question 1.2: Assigning Edge Types
      • Heterogeneous Graph Visualization
      • Transforming to DeepSNAP representation
      • Question 1.3: How many nodes are of each type (10 Points)
      • Question 1.4: Message Types - How many edges are of each message type (10 Points)
      • Question 1.5: Dataset Splitting - How many nodes are in each dataset split? (10 Points)
      • DeepSNAP Dataset Visualization
    • 2) Heterogeneous Graph Node Property Prediction
      • Setup
      • Dataset
      • Implementing HeteroGNNConv
      • Heterogeneous GNN Wrapper Layer
      • Initialize Heterogeneous GNN Layers
      • HeteroGNN
      • Training and Testing
      • Dataset and Preprocessing
      • Start Training!
      • Training the Mean Aggregation
      • Question 2.1: What is your maximum test set micro F1 score for the best_model when using mean aggregation? (10 points)
      • Question 2.2: What is your maximum test set macro F1 score for the best_model when using the mean aggregation? (10 points)
      • Training the Attention Aggregation
      • Question 2.3: What is your maximum test set micro F1 score for the best_model when using the attention aggregation? (4 points)
      • Question 2.4: What is your maximum test set macro F1 score for the best_model when using the attention aggregation? (4 points)
      • Attention for each Message Type
    • Submission

PyTorch_Geometric

  • 1. Introduction: Hands-on Graph Neural Networks
    • Implementing Graph Neural Networks
      • Embedding the Karate Club Network
      • Training on the Karate Club Network
    • Conclusion
  • 2. Node Classification with Graph Neural Networks
    • Training a Multi-layer Perception Network (MLP)
    • Training a Graph Neural Network (GNN)
    • Conclusion
    • (Optional) Exercises
  • 3. Graph Classification with Graph Neural Networks
    • Mini-batching of graphs
    • Training a Graph Neural Network (GNN)
    • (Optional) Exercise
    • Conclusion
  • 4. Scaling Graph Neural Networks
    • Conclusion
  • 5. Point Cloud Classification with Graph Neural Networks
    • Data Handling
    • Point Cloud Generation
    • PointNet++
      • Phase 1: Grouping via Dynamic Graph Generation
    • Phase 2: Neighborhood Aggregation
    • Network Architecture
    • Training Procedure
    • (Optional) Exercises
      • 1. Rotation-invariant PointNet Layer
      • 2. Downsampling Phase via Farthest Point Sampling
  • 6. Explaining GNN Model Predictions using Captum
    • Loading the dataset
    • Visualizing the data
    • Sample visualization
    • Training the model
      • Define the model
      • Define train and test functions
      • Train the model for 100 epochs
    • Explaining the predictions
  • 7. Customizing Aggregations within Message Passing with torch_geometric.nn.aggr
    • Loading the dataset
    • Define train, test and run functions
    • Training GNNs with torch_geometric.nn.aggr package
      • Define a GNN class
      • Original interface with string type as aggregation argument
      • Use a single aggregation module as aggregation argument
      • Use a list of aggregation strings as aggregation argument
      • Use a list of aggregation modules as aggregation argument
      • Use a list of mixed modules and strings as aggregation argument
      • Define multiple learnable aggregations with keyword arguments
      • Define multiple aggregations with MultiAggregation module
    • Conclusion
  • 8. Node Classification with W&B
    • Node Classification with Graph Neural Networks
    • Training a Multi-layer Perception Network (MLP)
    • Training a Graph Neural Network (GNN)
    • Using W&B Sweeps
    • Conclusion
    • (Optional) Exercises

Pyg_Tutorial_Project

  • Tutorial 1: Introduction
    • Load the dataset
      • Dataset properties
      • Dataset shapes
  • Tutorial 2: PyTorch basics
    • Outline
      • Imports
    • Datasets
      • A raw dataset
      • PyTorch Dataset
      • PyTorch DataLoader
    • Models
      • Model inspection
      • The forward method
    • Losses
      • The MSE loss
      • Giving a score to the model (parameters) given the training set
    • Optimization
      • Gradients
      • Handmade optimization
      • Stochastic GD
      • PyTorch optim
  • Tutorial 3: GAT implementation
    • Outline
    • Structure
    • Let’s start from the forward method
      • Linear Transformation
      • Attention Mechanism
      • Masked Attention
        • h_prime vs h
    • Build the layer
    • Use it
  • Tutorial 4: Convolutional Layers - Spectral methods
    • Outline
    • Imports
    • Why convolution in ML
    • Some theory on convolution
      • Definition
      • Fourier transform
      • Definition of the Fourier transform
      • Connection with the Laplacian
      • Summary
    • Convolution on graphs
      • Graph Laplacian
      • Graph spectrum, Fourier transform, and convolution
    • Spectral-convolutional layers in PyTorch Geometric
      • ChebConv
        • Goal:
        • Chebyshev approximation
        • Chebyshev approximation of the filter
        • Property
        • Fast approximated convolution
        • Properties:
      • GCNConv
        • As message passing
  • Tutorial 5: Aggregation
    • Message Passing Class
      • LAF Aggregation Module
      • PNA Aggregation
      • Test the new classes
  • Tutorial 6: GAE & VGAE
    • Graph AutoEncoder GAE
      • Load the data
      • Define the Encoder
      • Define the Autoencoder
      • Are the results (AUC) and (AP) easy to read and compare?
    • Use Tensorboard
      • Import tensorboard
        • Installation: (if needed) “pip install tensorboard”
      • Graph Variational AutoEncoder (GVAE)
  • Tutorial 7: ARGA & ARVGA
    • Imports
    • Define the dataset
    • Define the model
    • Initialize the model
      • Train the model
  • Tutorial 9_1: Recurrent GNNs
    • Graph Neural Network Model
      • Gated Graph Neural Network
  • Tutorial 9_2: Recurrent GNNs
    • Graph Neural Network Model
    • Gated Graph Neural Network
  • Tutorial 11: DeepWalk and node2vec - Implementation details
    • Setup
    • Random walks
      • The data loader
      • Visualization
    • Training
      • Model definition
      • Training function
      • Test function
      • Training
      • Visualization
  • Tutorial 12: GAE for link prediction
    • Simple autoencoder model
  • Tutorial 13: Node2Vec for link prediction
    • Step 1
      • The dataset
      • Use only one moleculas compound
      • Build the graph with
    • Step 2
      • Visualize node embedding
    • Step 3
      • Visualize edge embedding
    • Step 4
  • Tutorial 14: Data Handling in PyG (Part 1)
    • Data
      • Batch
      • Cluster
      • Sampler
      • Datasets
  • Tutorial 15: Data Handling in PyG (Part 2)
    • Custom PyG dataset
      • Open Graph Benchmark datasets
      • Benchmarking Graph Neural Networks Datasets

Category_theory

  • Definitions
    • Yoneda lemma
      • 1.8
      • 1.10
      • 1.13
      • 1.16
      • 1.17
      • 1.23
      • 1.25.
      • 1.31.
      • 1.54
      • 1.70.
      • 1.76
      • 1.87
      • 1.88
      • 1.90
      • 1.112

GDL_book

  • Ch 3 Geometric Priors
    • 3.1 Symmetries, Representations, and Invariance
      • Symmetry groups
        • group axioms
      • Group Actions and Group Representations
      • Invariant and Equivariant functions
    • 3.2 Isomorphisms and Automorphisms
      • Subgroups and Levels of structure
      • Isomorphisms and Automorphisms
    • 3.3 Deformation Stability
      • Stability to signal deformations
      • Stability to domain deformations
    • 3.4 Scale Separation
      • Fourier Transform and Global invariants
      • Multiscale representations
      • Deformation stability of Multiscale representations
      • Scale Separation Prior
    • 3.5 The Blueprint of Geometric Deep Learning
      • Different settings of Geometric Deep Learning
  • Ch 4 Geometric Domains: the 5 Gs
    • 4.1 그래프 및 세트
    • 4.2 그리드와 유클리드 공간
      • 순환 행렬 및 컨볼루션
      • 이산 푸리에 변환의 유도
      • 연속 푸리에 변환의 유도
    • 4.3 그룹 및 동질 공간
      • 그룹 컨볼루션
      • 구형 컨볼루션
    • 4.4 측지 및 매니폴드
      • 리만 다양체
      • 스칼라 및 벡터 필드
      • 내재적 그라데이션
      • 측지학
      • 병렬 전송
      • 지수 맵
      • 측지 거리
      • 아이소메트리
      • 내재적 대칭
      • 매니폴드에 대한 푸리에 분석
      • 다양체의 스펙트럼 컨볼루션
      • 다면체의 공간 컨볼루션
    • 4.5 게이지 및 번들
      • 탄젠트 번들 및 구조 그룹
      • 게이지 대칭
    • 4.6 기하학적 그래프 및 메시
      • 라플라시안 행렬
      • 메시의 스펙트럼 분석
      • 연산자 및 함수형 맵으로서의 메시
  • Ch 5 Geometric Deep Learning Models
    • 5.1 컨볼루션 신경망
      • 효율적인 멀티스케일 컴퓨팅
      • 심층 및 잔여 네트워크
      • 정규화
      • 데이터 증강
    • 5.2 그룹 등변량 CNN
      • 이산 그룹 컨볼루션
      • 트랜스폼+컨볼브 접근 방식
      • 푸리에 영역의 구형 CNN
    • 5.3 그래프 신경망
    • 5.4 딥 세트, 트랜스포머 및 잠재 그래프 추론
      • 빈 가장자리 세트
      • 완전한 엣지 세트
      • 추론된 에지 세트
    • 5.5 등변량 메시지 전달 네트워크
      • 환원 불가능한 표현
      • 일반 표현
    • 5.6 고유 메시 CNN
      • 측지 패치
      • 등방성 필터
      • 고정 게이지
      • 앵귤러 풀링
      • 게이지 등변량 필터
    • 5.7 반복 신경망
      • 단순한 RNN
      • RNN의 번역 등가성
      • RNN의 깊이
    • 5.8 장단기 메모리 네트워크
      • 게이트 RNN의 시간 왜곡 불변성
      • RNN을 사용한 시퀀스 간 학습

Papers

  • Message passing all the way up
    • Feature augmentation
    • Message passing modulation
    • Graph rewiring
    • Subgraph aggregation
    • Substructure based methods
    • General equivariant GNNs
    • General equivariant GNNs

AMMI_ex

  • Ex 1

Notes

  • Chemprop
    • Molecular Property Prediction
    • Analyzing Learned Molecular Representations for Property Prediction
      • Takeaways
    • A Deep Learning Approach to Antibiotic Discovery
      • Features
      • Additional molecule-level features
      • Ensembling
    • Machine Learning of Reaction Properties via Learned Representations of the Condensed Graph of Reaction
  • Comparison of One-Hot Encoding and Embedding Layer
    • One-Hot Encoding
      • Pros of One-Hot Encoding:
      • Cons of One-Hot Encoding:
    • Embedding Layer
      • Pros of Embedding Layer:
      • Cons of Embedding Layer:
    • When to Use Which?
      • Example
  • Cumulative Density Functions (CDFs)
    • Definition:
    • Pros:
    • Cons:
    • Examples:
  • Gumbel Distribution and Gumbel-Softmax
    • Gumbel Distribution
    • Gumbel-Softmax
    • Advantages and Disadvantages
      • Advantages
      • Disadvantages
    • Relation to Machine Learning
  • Kullback-Leibler Divergence
    • KL Divergence in Machine Learning
  • Lec_7_23 Shift operator
  • Lie algebras and Lie groups
  • Pooling Architecture Search
    • Differentiable Search Algorithm
      • Pros
      • Cons
  • Tanimoto Similarity Explained
  • The sources of error
  • One label per class
  • Scaffold-based Split
    • Definition
    • Pros
    • Cons
    • Comparison to Other Methods
    • Example in Python
  • Stratified Split
    • Definition
    • Pros
    • Cons
    • Comparison to Other Methods
    • Example in Python
  • Supernet
rosicast
  • »
  • Search

© Copyright 2022, rosicast. Revision 35f8cba8.

Built with Sphinx using a theme provided by Read the Docs.