Skip to main content

2 posts tagged with "o2"

View All Tags

Improving Code Productivity with Design Patterns in O2

· 7 min read
Haril Song
Owner, Software Engineer at 42dot

This article discusses the process of using design patterns to improve the structure of the O2 project for more flexible management.

Problem

While diligently working on development, a sudden Issue was raised one day.

image

Reflecting the contents of the Issue was not difficult. However, as I delved into the code, an issue that had been put off for a while started to surface.

image

Below is the implementation of the Markdown syntax conversion code that had been previously written.

warning

Due to the length of the code, a partial excerpt is provided. For the full code, please refer to O2 plugin v1.1.1 🙏

export async function convertToChirpy(plugin: O2Plugin) {
try {
await backupOriginalNotes(plugin);
const markdownFiles = await renameMarkdownFile(plugin);
for (const file of markdownFiles) {
// remove double square brackets
const title = file.name.replace('.md', '').replace(/\s/g, '-');
const contents = removeSquareBrackets(await plugin.app.vault.read(file));
// change resource link to jekyll link
const resourceConvertedContents = convertResourceLink(plugin, title, contents);

// callout
const result = convertCalloutSyntaxToChirpy(resourceConvertedContents);

await plugin.app.vault.modify(file, result);
}

await moveFilesToChirpy(plugin);
new Notice('Chirpy conversion complete.');
} catch (e) {
console.error(e);
new Notice('Chirpy conversion failed.');
}
}

Being unfamiliar with TypeScript and Obsidian usage, I had focused more on implementing features rather than the overall design. Now, trying to add new features, it became difficult to anticipate side effects, and the code implementation lacked clear communication of the developer's intent.

To better understand the code flow, I created a graph of the current process as follows.

Although I had separated functionalities into functions, the code was still procedurally written, where the order of code lines significantly impacted the overall operation. Adding a new feature in this state would require precise implementation to avoid breaking the entire conversion process. So, where would be the correct place to implement a new feature? Most likely, the answer would be 'I need to see the code.' Currently, with most of the code written in one large file, it was almost equivalent to needing to analyze the entire code. In object-oriented terms, one could say that the Single Responsibility Principle (SRP) was not being properly followed.

This state, no matter how positively described, did not seem easy to maintain. Since the O2 plugin was created for my personal use, I could not justify producing spaghetti code that was difficult to maintain by rationalizing, 'It's because I'm not familiar with TS.'

Before resolving the Issue, I decided to first improve the structure, putting the glory aside for a moment.

What Structure Should Be Implemented?

The O2 plugin, as a syntax conversion plugin, must be capable of converting Obsidian's Markdown syntax into various formats, which is a clear requirement.

Therefore, the design should focus primarily on scalability.

Each platform logic should be modularized, and the conversion process abstracted to implement it like a template. This way, when implementing new features for supporting different platform syntaxes, developers can focus only on the small unit of implementing syntax conversion without needing to reimplement the entire flow.

Based on this, the design requirements are as follows:

  1. Strings (content of Markdown files) should be converted in order (or not) as needed.
  2. Specific conversion logic should be skippable or dynamically controllable based on external settings.
  3. Implementing new features should be simple and should have minimal or no impact on existing code.

As there is a sequence of execution, and the ability to add features in between, the Chain of Responsibility pattern seemed appropriate for this purpose.

Applying Design Patterns

Process->Process->Process->Done! : Summary of Chain of Responsibility

export interface Converter {
setNext(next: Converter): Converter;
convert(input: string): string;
}

export abstract class AbstractConverter implements Converter {
private next: Converter;

setNext(next: Converter): Converter {
this.next = next;
return next;
}

convert(input: string): string {
if (this.next) {
return this.next.convert(input);
}
return input;
}
}

The Converter interface plays a role in converting specific strings through convert(input). By specifying the next Converter to be processed with setNext, and returning the Converter again, method chaining can be used.

With the abstraction in place, the conversion logic that was previously implemented in one file was separated into individual Converter implementations, assigning responsibility for each feature. Below is an example of the CalloutConverter that separates the Callout syntax conversion logic.

export class CalloutConverter extends AbstractConverter {
convert(input: string): string {
const result = convertCalloutSyntaxToChirpy(input);
return super.convert(result);
}
}

function convertCalloutSyntaxToChirpy(content: string) {
function replacer(match: string, p1: string, p2: string) {
return `${p2}\n{: .prompt-${replaceKeyword(p1)}}`;
}

return content.replace(ObsidianRegex.CALLOUT, replacer);
}

Now, the relationships between the classes are as follows.

Now, by combining the smallest units of functionality implemented in each Converter, a chain is created to perform operations in sequence. This is why this pattern is called Chain of Responsibility.

export async function convertToChirpy(plugin: O2Plugin) {
// ...
// Create conversion chain
frontMatterConverter.setNext(bracketConverter)
.setNext(resourceLinkConverter)
.setNext(calloutConverter);

// Request the frontMatterConverter at the head to perform the conversion, and the connected converters will operate sequentially.
const result = frontMatterConverter.convert(await plugin.app.vault.read(file));
await plugin.app.vault.modify(file, result);
// ...
}

Now that the logic has been separated into appropriate responsibilities, reading the code has become much easier. When needing to add a new feature, only the necessary Converter needs to be implemented. Additionally, without needing to know how other Converters work, new features can be added through setNext. Each Converter operates independently, following the principle of encapsulation.

Finally, I checked if all tests passed and created a PR.

image

Next Step

Although the structure has improved significantly, there is one remaining drawback. In the structure linked through setNext, calling the Converter at the very front is necessary for proper operation. If a different Converter is called instead of the one at the front, the result may be different from the intended one. If, for example, a NewConverter is implemented before frontMatterConverter but frontMatterConverter.convert(input) is not modified, NewConverter will not be applied.

This is one of the aspects that developers need to pay attention to, as there is room for error, and it is one of the areas that needs improvement in the future. For instance, implementing a kind of Context to contain the Converters and executing the conversion process without directly calling the Converters could be a way to improve. This is something I plan to implement in the next version.


2023-03-12 Update

Thanks to PR, the same functionality was performed, but with a more flexible structure using composition instead of inheritance.

Conclusion

In this article, I described the process of redistributing roles and responsibilities through design patterns from a procedurally written, monolithic file to a more object-oriented and maintainable code.

info

The complete code can be found on GitHub.

[O2] Developing an Obsidian Plugin

· 7 min read
Haril Song
Owner, Software Engineer at 42dot

Overview

Obsidian provides a graph view through links between Markdown files, making it convenient to store and navigate information. However, to achieve this, Obsidian enforces its own unique syntax in addition to the original Markdown syntax. This can lead to areas of incompatibility when reading Markdown documents from Obsidian on other platforms.

Currently, I use a Jekyll blog for posting, so when I write in Obsidian, I have to manually adjust the syntax later for blog publishing. Specifically, the workflow involves:

  • Using [[]] for file links, which is Obsidian's unique syntax
  • Resetting attachment paths, including image files
  • Renaming title.md to yyyy-MM-dd-title.md
  • Callout syntax

image Double-dashed arrows crossing layer boundaries require manual intervention.

As I use both Obsidian and Jekyll concurrently, there was a need to automate this syntax conversion process and attachment copying process.

Since Obsidian allows for functionality extension through community plugins unlike Notion, I decided to try creating one myself. After reviewing the official documentation, I found that Obsidian guides plugin development based on NodeJS. While the language options were limited, I had an interest in TypeScript, so I set up a NodeJS/TS environment to study.

Implementation Process

Naming

I first tackled the most important part of development.

It didn't take as long as I thought, as I came up with the project name 'O2' based on a sudden idea while writing a description, 'convert Obsidian syntax to Jekyll,' for the plugin.

image

Preparation for Conversion

With a suitable name in place, the next step was to determine how to convert which files.

The workflow for blog posting is as follows:

  1. Write drafts in a folder named ready.
  2. Once the manuscript is complete, copy the files, including attachments, to the Jekyll project, appropriately converting Obsidian syntax to Jekyll syntax in the process.
  3. Move the manuscript from the ready folder to published to indicate that it has been published.

I decided to program this workflow as is. However, instead of editing the original files in a Jekyll project open in VScode, I opted to create and modify copies internally in the plugin workspace to prevent modification of the original files and convert them to Jekyll syntax.

To summarize this step briefly:

  1. Copy the manuscript A.md from /ready to /published without modifying /published/A.md.
  2. Convert the title and syntax of /ready/A.md.
  3. Move /ready/yyyy-MM-dd-A.md to the path for Jekyll publishing.

Let's start the implementation.

Copying Original Files

// Get only Markdown files in the ready folder
function getFilesInReady(plugin: O2Plugin): TFile[] {
return this.app.vault.getMarkdownFiles()
.filter((file: TFile) => file.path.startsWith(plugin.settings.readyDir))
}

// Copy files to the published folder
async function copyToPublishedDirectory(plugin: O2Plugin) {
const readyFiles = getFilesInReady.call(this, plugin)
readyFiles.forEach((file: TFile) => {
return this.app.vault.copy(file, file.path.replace(plugin.settings.readyDir, plugin.settings.publishedDir))
})
}

By fetching Markdown files inside the /ready folder and replacing file.path with publishedDir, copying can be done easily.

Copying Attachments and Resetting Paths

function convertResourceLink(plugin: O2Plugin, title: string, contents: string) {
const absolutePath = this.app.vault.adapter.getBasePath()
const resourcePath = `${plugin.settings.jekyllResourcePath}/${title}`
fs.mkdirSync(resourcePath, {recursive: true})

const relativeResourcePath = plugin.settings.jekyllRelativeResourcePath

// Copy resourceDir/image.png to assets/img/<title>/image.png before changing
extractImageName(contents)?.forEach((resourceName) => {
fs.copyFile(
`${absolutePath}/${plugin.settings.resourceDir}/${resourceName}`,
`${resourcePath}/${resourceName}`,
(err) => {
if (err) {
new Notice(err.message)
}
}
)
})
// Syntax conversion
return contents.replace(ObsidianRegex.IMAGE_LINK, `![image](/${relativeResourcePath}/${title}/$1)`)
}

Attachments require moving files outside the vault, which cannot be achieved using Obsidian's default APIs. Therefore, direct file system access using fs is necessary.

info

Direct file system access implies difficulty in mobile usage, so the Obsidian official documentation guides specifying isDesktopOnly as true in manifest.json in such cases.

Before moving Markdown files to the Jekyll project, the Obsidian image link syntax is parsed to identify image filenames, which are then moved to Jekyll's resource folder so that the Markdown default image links are converted correctly, allowing attachments to be found.

Callout Syntax Conversion

Obsidian callout

> [!NOTE] callout title
> callout contents

Supported keywords: tip, info, note, warning, danger, error, etc.

Jekyll chirpy callout

> callout contents
{: .promt-info}

Supported keywords: tip, info, warning, danger

As the syntax of the two differs, regular expressions are used to substitute this part, requiring implementation of a replacer.

export function convertCalloutSyntaxToChirpy(content: string) {
function replacer(match: string, p1: string, p2: string) {
if (p1.toLowerCase() === 'note') {
p1 = 'info'
}
if (p1.toLowerCase() === 'error') {
p1 = 'danger'
}
return `${p2}\n{: .prompt-${p1.toLowerCase()}}`
}

return content.replace(ObsidianRegex.CALLOUT, replacer)
}

Unsupported keywords in Jekyll are converted to other keywords with similar roles.

Moving Completed Files

The Jekyll-based blog I currently use has a specific path where posts need to be located for publishing. Since the Jekyll project location may vary per client, custom path handling is required. I decided to set this up through a settings tab and created an input form like the one below.

image

Once all conversions are done, moving the files to the _post path in Jekyll completes the conversion process.

async function moveFilesToChirpy(plugin: O2Plugin) {
// Absolute path is needed to move files outside the vault
const absolutePath = this.app.vault.adapter.getBasePath()
const sourceFolderPath = `${absolutePath}/${plugin.settings.readyDir}`
const targetFolderPath = plugin.settings.targetPath()

fs.readdir(sourceFolderPath, (err, files) => {
if (err) throw err

files.forEach((filename) => {
const sourceFilePath = path.join(sourceFolderPath, filename)
const targetFilePath = path.join(targetFolderPath, filename)

fs.rename(sourceFilePath, targetFilePath, (err) => {
if (err) {
console.error(err)
new Notice(err.message)
throw err
}
})
})
})
}

Regular Expressions

export namespace ObsidianRegex {
export const IMAGE_LINK = /!\[\[(.*?)]]/g
export const DOCUMENT_LINK = /(?<!!)\[\[(.*?)]]/g
export const CALLOUT = /> \[!(NOTE|WARNING|ERROR|TIP|INFO|DANGER)].*?\n(>.*)/ig
}

Special syntax unique to Obsidian was handled using regular expressions for parsing. By using groups, specific parts could be extracted for conversion, making the process more convenient.

Creating a PR for Community Plugin Release

Finally, to register the plugin in the community plugin repository, I conclude by creating a PR. It is essential to adhere to community guidelines; otherwise, the PR may be rejected. Obsidian provides guidance on what to be mindful of when developing plugins, so it's crucial to follow these guidelines as closely as possible.

image

Based on previous PRs, it seems that merging takes approximately 2-4 weeks. If feedback is received later, I will make the necessary adjustments and patiently wait for the merge.

Conclusion

I thought, 'This should be a quick job, maybe done in 3 days,' but trying to implement the plugin while traveling abroad ended up taking about a week, including creating the release PR 😂

image I wonder if Kent Beck and Erich Gamma, who developed JUnit, coded like this on a plane...

Switching to TypeScript from Java or Kotlin made things challenging, as I wasn't familiar with it, and I wasn't confident if the code I was writing was best practice. However, thanks to this, I delved into JS syntax like async-await in detail, adding another technology stack to my repertoire. It's a proud feeling. This also gave me a new topic to write about.

The best part is that there's almost no need for manual work in blog posting anymore! After converting the syntax with the plugin, I only need to do a spell check before pushing to GitHub. Of course, there are still many bugs...

Moving forward, I plan to continue studying TypeScript gradually to eliminate anti-patterns in the plugin and improve the design for cleaner modules.

If you're facing similar dilemmas, contributing to the project or collaborating in other ways to build it together would be great! You're welcome anytime 😄

info

You can check out the complete code on GitHub.

Next Steps 🤔

  • Fix minor bugs
  • Support footnote syntax
  • Support image resize syntax
  • Implement transaction handling for rollback in case of errors during conversion
  • Abstract processing for adding other modules

Release 🚀

After about 6 days of code review, the PR was merged. The plugin is now available for use in the Obsidian Community plugin repository. 🎉

image

Reference